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FISCAL INFORMATION 



4. Account Number: 
13950-55500. 



6. Total amount this action: 

$45,390,092.00 



8. Revenue generated this action: 
$0.00 



5. Account Name: 
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32. If yes, list the %: 

Minority: 

Women: 0,7.. 
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Convenience" clause in the 
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IC 20-19-3-5 



37. Description of work and justification for spending money. (Please give a brief description of the scope of work included in this agreement.) 

CTB/McGraw-Hill will provide all needed services required to deliver ISTEP+ assessments for students in grades 3-8, including design, development and implementation of testing 
and scoring services. 



38. Justification of vendor selection and determination of price reasonableness: 



Through competition coordinated by the Indiana Department of Administration , CTB/McGraw-Hill was the highest scoring (winning) vendor for Component 3 of RFBJ-93 and was , , 

ides 3-8. Based on the proposals submitted,' the --' — " >' 'U- jfl }J 



therefore selected to provide Component 3 of Indiana's assessment system, which includes ISTEP+ assessments for students in grades 
price is considered reasonable for the services provided. 
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CTB/McGraw-Hill will provide all needed services required to 
deliver ISTEP+ assessments for students in grades 3-8, 
including design, development and implementation of testing 
and scoring services. » 
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Common Codes are used In this document: 
MON Month 



Requisition No. Date Required Date Page 
0000007313 06/15/2012 1 of 1 
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Project Number: 

Requisition Number: 0000007313 
Requestor: M254188 Melissa Ann Covey 
Agency Number: 00700 Department of Education 
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?YV Covey. 
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Authorized Signature 
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AMENDMENT #1 
EDS#A58-MlSA-005 



This is an amendment to the Contract (the "Contract") entered into by and between the Indiana 
Department of Education (the "State") and CTB / McGraw-Hill (the "Contractor") dated July 1, 2010 
through June 30, 2012. 

In consideration of the mutual undertakings and covenants hereinafter set forth, the parties agree as 
follows: 

The Contract is hereby amended for an additional period of two years. It shall terminate on 
June 30, 2014. 

This amendment provides an increase of $45,390,092 as set forth in Exhibit G attached and 
herein incorporated by reference. Total remuneration under the Contract is not to exceed 
$95,039,790. 

The following clauses in the Contract are amended as follows: 
1. Duties of Contractor 

Paragraph 1, relating to Duties of the Contractor is hereby amended by addition of the following: 

The Contractor will provide all needed services required to deliver ISTEP+ assessments for 
students in grades 3-8, including design, development and implementation of testing and scoring 
services. Details of duties are described in Exhibit E and Exhibit F attached herein and incorporated 
by reference. 



2. Consideration 

deleted in its entirety and replaced by the following: 
Paragraph 2, relating to Consideration is hereby a m e nded by the addition of the following: 

The Contractor will be paid in accordance with the billing schedule described in Exhibit C of the 
original contract attached and herein incorporated by reference and Exhibit G attached and herein 
incorporated by reference for performing duties set forth above. Total remuneration under this 
Contract shall not exceed Ninety-five million Thirty-nine Thousand Seven Hundred Ninety 
Dollars ($95,039,790). 

3. Term 



Paragraph 3, relating to Term is hereby deleted in its entirety and replaced by the follows: following: 



This contract shall be effective for a period of four years. It commenced on July 1, 2010 and 
shall remain in effect through June 30, 2014. 
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11. Confidentiality of State Information 

Paragraph 11, relating to Confidentiality of State Information is hereby amended by addition of the 
following: 

Security and Privacy of Student Information 

The Contractor understands that in the performance of this Contract, the Contractor will be provided 
with personally identifiable student information subject to the protections of the Family Educational 
Rights and Privacy Act ("FERPA") (20 USC § 1232(g), the National School Lunch Act (42 USC 
§1751 et. seq.), the Child Nutrition Act (42 USC §1771 et. seq.), the regulations implementing those 
Acts, and 511 IAC 7-38. As a result, the Contractor is bound by the security provisions and 
confidentiality obligations required by these laws and regulations. 

1 . Except as permitted by this Contract, the Contractor agrees not to share or re-disclose personally 
identifiable data, including de-identified data, received under this Contract with any other entity, 
organization, or individual. 

2. Despite any contraiy provisions in this Contract, the Contractor agrees not to disclose any 
personally identifiable data obtained in a manner that could identify, or reasonably lead to the 
identification of, an individual student to any other individual, institution, organization, 
government, or entity. This includes data that may not have been provided in a personally 
identifiable manner, but would still permit public identification of students due to small cell sizes 
(i.e., subgroups of 10 or fewer students). Such data and data elements may only be used in the 
aggregate or other manner consistent with generally accepted statistical principles that does not 
permit identification of students. 

3. The Contractor shall promptly, that is, within one (1) hour of learning, report to the State any 
incidents in detail of any personally identifiable information received from the State whose 
confidentiality was breached, released, or is believed to have been breached or released. 

4. The Contractor agrees to destroy all personally identifiable and de-identified information received 
under this Contract when it is no longer needed for the purpose for which it was obtained. The 
Contractor shall maintain records that document and verity the destruction of the data provided 
by the State pursuant to this Contract. If the Contractor does not destroy personally identifiable 
information pursuant to the terms of this Contract, when no longer needed for the purposes of this 
Contract, then the Contractor may be prohibited from access to future personally identifiable 
information from educational records for at least five (5) years. 

5. Subject to the agreed-upon and limited use of the data and only for the purposes asserted in this 
Contract, there shall be no further disclosure by the Contractor of any of the information provided 
by the State in that this would constitute a redisclosure of information. Under the applicable 
federal FERPA regulation, redisclosure is only permitted upon obtaining prior consent of the 
parent or eligible student of the personally identifiable information. 



47. Liquidated Damages 

Paragraph 47, relating to Liquidated Damages Clause is hereby amended by addition of the following: 
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General timelines for deliverables and services under this Contract are set forth in Exhibit E 
(Scope of Work) attached hereto and herein incorporated. 



A. It is hereby agreed that, without limitation to other rights and remedies of the State under 
this Contract, the State shall be entitled to liquidated damages for the Contractor's failure 
to deliver, pursuant to the provisions of this Contract and Key Performance Dates, 
defined in Exhibit F (Deliverables Overview) attached hereto and herein incoiporated, 
any of the following deliverables: 

• Paper testing materials delivery; 

• Print reports delivery; 

• "INORS" summary reports delivery; 

• "Parent Network" reports delivery; and 

• "OAS" online testing delivery, which is defined as uninterrupted CTB system 
availability every school day from 7:00 a.m. to 6:00 p.m. Eastern Time for the 
two weeks prior to each testing window and for the entire testing window 

in accordance with the Key Performance Dates for each year of the Contract as specified 
in Exhibit F hereto. In the event of the Contractor's failure, liquidated damages shall 
begin to accrue the calendar day of such failure of Contractor to perform. The State shall 
be entitled to liquidated damages for each deliverable listed above in the amount of 
$50,000 per day for each of the first five (5) calendar days of the Contractor's failure to 
deliver, $150,000 per day for each of the next five (5) calendar days (days 6-10) and 
$250,000 per day for each calendar day thereafter until such time that the Contractor 
cures the failure to deliver. The liquidated damages for any single delivery failure shall 
not exceed 10% of the current contract amount. 

B. It is hereby agreed further that, without limitation to other rights and remedies of the 
State under this Contract, the State shall be entitled to liquidated damages for the 
Contractor's failure to deliver all other deliverables or complete all tasks pursuant to the 
Contract and not otherwise specified in Paragraph A above, in accordance with the Key 
Performance Dates for each year of the Contract as specified in Exhibit F (Deliverables 
Overview). The State shall provide the Contractor with written notice of the Contractor's 
failure to perform, and the Contractor shall have five (5) calendar days from receipt of the 
written notice to cure the failure without penalty. In the event that the Contractor fails to 
cure within the prescribed time period, liquidated damages shall begin to accrue, and the 
State shall be entitled to liquidated damages in the amount of $25,000 per day for each of 
the first five (5) calendar days after the time period for curing without penalty has elapsed 
and $100,000 per day for each calendar day thereafter until such time that the Contractor 
cures the failure to deliver or otherwise perform. The liquidated damages for any single 
delivery or performance failure shall not exceed 10% of the current contract amount. 

C. It is agreed that liquidated damages will not accrue against the Contractor for failure to 
meet an original Key Performance Date if, prior to the specific Key Performance Date, 
the parties hereto mutually agree upon a change of date for a Key Deliverable, and an 
amendment incorporating the change of date is executed by the parties to the Contract. 
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However, liquidated damages as described in Paragraph A and B of this section, are 
applicable should the Contractor fail to perform in accordance with the amended Key 
Performance Dates deadline(s). 

D. It is agreed that, should the Contractor be prevented from timely performing its 
obligations hereunder by a force majeure event, including the State's acts or failure to act 
in its sovereign and contractual capacity such as a change in the scope of work or the 
failure to provide required approvals, the Contractor's time to perform shall be extended 
for the period of delay resulting from such event(s) as determined by the State. 

E. In respect of any Key Performance Dates, the State shall not be entitled to receive from 
the Contractor liquidated damages pursuant to this Contract or otherwise in excess of the 
amounts specified herein. 

46. State Boilerplate Affirmation Clause 

Paragraph 46, relating to State Boilerplate Affirmation Clause is hereby amended by addition of the 
following: 

48. Employment Eligibility Verification (Clause Added) 
48. Employment Eligibility Verification 

The Contractor affirms under the penalties of perjury that he/she/it does not knowingly employ an 
unauthorized alien. 

The Contractor shall enroll in and verify the work eligibility status of all his/her/its newly hired 
employees through the E-Verify program as defined in IC 22-5-1 .7-3. The Contractor is not required to 
participate should the E-Verify program cease to exist. Additionally, the Contractor is not required to 
participate if the Contractor is self-employed and does not employ any employees. 

The Contractor shall not knowingly employ or contract with an unauthorized alien. The Contractor shall 
not retain an employee or contract with a person that the Contractor subsequently learns is an 
unauthorized alien. 

The Contractor shall require his/her/its subcontractors, who perform work under this contract, to certify to 
the Contractor that the subcontractor does not knowingly employ or contract with an unauthorized alien 
and that the subcontractor has enrolled and is participating in the E-Verify program. The Contractor 
agrees to maintain this certification throughout the duration of the term of a contract with a subcontractor. 

The State may terminate for default if the Contractor fails to cure a breach of this provision no later than 
thirty (30) days after being notified by the State. 

All other matters previously agreed to and set forth in the original Contract and not affected by this 
Amendment, shall remain in full force and effect. 



THE REMAINDER OF THIS PAGE HAS BEEN INTENTIONALLY LEFT BLANK. 
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Non-Collusion and Acceptance 



The undersigned attests, subject to the penalties for perjury, that he/she is the Contractor, or that he/she is the 
properly authorized representative, agent, member or officer of the Contractor, that he/she has not, nor has 
any other member, employee, representative, agent or officer of the Contractor, directly or indirectly, to the 
best of the undersigned's knowledge, entered into or offered to enter into any combination, collusion or 
agreement to receive or pay, and that he/she has not received or paid, any sum of money or other 
consideration for the execution of this Contract other than that which appears upon the face of this Contract. 

In Witness Whereof, Contractor and the State have, through their duly authorized representatives, 
entered into this Contract. The parties, having read and understood the foregoing terms of this Contract, 
do by their respective signatures dated below hereby agree to the terms thereof. 



CTB/McGraw-Hill 




Printed Name: ^A/VQog. 
Title: Grt/g/r OPtLKAT/rtr~ 0PFiC£Lf^ ''chizf FinaHC/au C~>F=F/Cjf:tZ- 



Date: 



Indiana Department of Education 



By 



Dr. Tony Bennett 
Superintendent of Public Instruction 



Date: 



Indiana Office of Technology 



iod, Chieflnformati 



Department of Adrni 



By:. 

Brian Arrowood, ChiefTnformation Officer 
Date: 7"~^~"" /2- 

State Budget Agdncy 



?/J/,-4^D ( fort 

yUt^ M ■ Chotf^&^Mrf, Director 
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Exhibit E 



SCOPE OF WORK 



CTB McGraw-Hill 
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EXHIBIT E 
SCOPE OF WORK 

NOTE: Within this Exhibit E. references to "CTB" and "McGraw-Hill, " indicate the "Conductor": 
references to "IDOE" indicate the Indiana State Department of Education and references to "ISTEP+" 
indicates the Project 's name under this Contract. 
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Executive Summary 

This scope of work document provides detailed information and specification for the 2012-2014 ISTEP+ 
program. This plan incorporates and expands on elements from prior ISTEP+ contracts. 

The ISTEP+ 2012-2014 Scope of Work includes Grades 3 through 8 Spring 2013 and Spring 2014 test 
administrations that will test E/LA and Math for Grades 3-8, Science for Grades 4 and 6, and Social Studies 
for Grades 5 and 7. 

With the exception of the Student Label, one copy each of the Invitation Code letter and Individual Student 
Report, all reports will be provided online. 

There will be two testing windows, one for Applied Skills (constructed-response items to be administered in 
early March) and another for Multiple-Choice (late April/early May). 

The operational testing window for each year includes: 

Spring 2013 

• Approximately 1 0% of the total student population will take the Applied Skills test online through the 
OAS application for the two content areas, E/LA and Social Studies, with the remainder (90%) 
taking the paper/pencil tests. As a contingency to the student readiness with online testing Applied 
Skills portions of the test, just for this administration, 100% of paper tests will be printed and be made 
available if needed. 

• Applied Skills Mathematics and Science content areas will be administered 1 00% in paper/pencil 
mode only. 

• At least 65% of the total student population will take all content areas of the Multiple-Choice test 
online through the OAS application. 

Spring 2014 

• Approximately 20% of the total student population will take the Applied Skills test online through the 
OAS application for the two content areas, E/LA and Social Studies, with the remainder (80%) 
taking the paper/pencil tests. 

• Applied Skills Mathematics and Science content areas will be administered 100% in paper/pencil 
mode only. 

• At least 80% of the student population will take all content areas of the Multiple-Choice test online 
through the OAS application. 

hi addition to the operational testing, pilot tests will be administered in the year 2013 and 2014 as noted 
below: 

Spring 2013 

• Applied Skills pilot forms based on Indiana Academic Standards will be administered to select 
population for all grades in ELA, Mathematics and Social Studies content areas. These pilot forms 
will be administered using paper/pencil mode, and also using online mode if student readiness and 
site participation is confirmed for Applied Skills online testing. 

• Multiple-Choice pilot forms based on Common Core State Standards (CCSS) will be administered to 
a select population for all grades in ELA and Mathematics content areas, using the online mode only. 
The Online Multiple-Choice pilot tests will assess and provide a transition to the state's readiness in 
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implementing the CCSS and will use CTB-owned CoreLink items. All future references to this pilot 
test will be referenced as 'CCSS Transition Online pilot ' in this document. 

Spring 2014 

• CCSS Transition Online pilot forms from 2013 will be repeat administered to a select population for 
all grades in ELA and Mathematics content areas, using the online mode only. 

There is a 5-calendar day turnaround time for operational testing reports after the receipt of the Multiple- 
Choice portion of the test (late April/early May). For significant changes to the percent of the students testing 
online for Multiple-Choice, the turnaround time will be negotiated and agreed upon with the IDOE. 

For the Spring 2013 administration only, a Cutscore Setting meeting for the Science content area will occur 
after the Spring 2013 test administration is completed. Because of this Cutscore Setting activity for Science, 
reports for E/LA, Math and Social Studies will be released following the 5-calendar day turnaround time, 
while for Science, reports will be released in Summer 2013. 

Based on the data collected from the CCSS Transition Online pilot tests and subsequent research analysis, 
two new reports will be designed and reported at the State and Corporation levels to provide information 
relating to the CCSS. These new reports will not include student level data and will be reported outside of the 
regular operational reporting windows. 
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Program Contacts and Communication 



Table 1 - Key IDOE Staff 



Name 


Position 


Telephone 


Email 


Wcs Bruce 


Chief Assessment Officer 


317-232-9054 


\vbruce(S>.doc. in.gov 


Michele Walker 


Director of Student Assessment 


317-232-9051 


mwalkertSJdoe.in.gov 


Charity Flores 


Assistant Director of Student 
Assessment 


317-234-5595 


cfloresfS),doe.in.eov 


Linda Potter 


Assessment Support Specialist 


317-232-9056 


lpottcr(S>.doe.in.fiov 


Sharon Watts 


Assessment Support Specialist 


317-234-5585 


s\vatts(S)doe.in.gov 


Jim Mirabelli 


Mathematics Assessment Specialist 


317-234-5602 


imirabeI(S),doc.in.Rov 


Ben Kemp 


Mathematics and Science Assessment 
Specialist 


317-234-1333 


bkcmDfSl.doe.in.gov 


Kelly Connelly 


E/LA and Social Studies Assessment 
Specialist 


317-234-5598 


kconncllv(S)doe.in.eov 


Erin Thompson 


ELA Assessment Specialist 


317-234-5599 


ethompsonfSldoe. i n . eo v 


Table 2 -Key CTB Staff 


Name 


Position 


Telephone 


Email 


Anu Tummala 


Senior Director, Program and Project 
Management 


831-393-7622 


Anu tummala(S).ctb.com 


Shiva Doreswamy 


Senior Program Manager 


831-393-7135 


sdores\vamy(S),ctb.com 


Vacant 


Program Manager 






Cece Robinson 


Program Office Coordinator 


831-393-7195 


Cece robinsonfSjctb.com 


Lisa Staalenburg 


Program Schedule Analyst 


831-393-6855 


Lisa staalenbure(S)ctb.com 


Linn Williams 


State Solutions Manager 


831-393-7896 


Linn williamsfSjctb.com 


Jerry McCanna 


Field Representative 


317-581-9404 


Jerrv mccanna(S).ctb.com 


Krista Bobbirt 


Content Development Manager 


805-466-2921 


Krista bobbitt(S)ctb.com 


Olga Connolly 


Content Development Lead 


850-875-3319 


Olga connolly(S),ctb.eom 


Karen Barton 


Research Principal 


919-386-0178 


Karen barton(S),ctb.com 


Dong-In Kim 


Research Scientist 


303-721-4214 


Donn-In KimfSictb.com 


Leslie Simon 


Production Coordinator 


831-393-7268 


Leslie simon(S),ctb.com 


Luna Kang 


Online Production Coordinator 


831-393-6554 


Luna kanafolctb.com 


Darin Homer 


Content Development Project Manager 


916-983-0809 


Darin homer(S).ctb.com 


Amanda Hill 


Scoring and Research Project Manager 


317-205-0220 


Amanda hillfSictb.com 


Susan Griffith 


Technology Project Manager 


831-393-6044 


Susan arifFithfSl.ctb.com 


Will Fredricks 


Manufacturing Specialist 


815-733-5097 


Will fredricksfS),ctb,com 


Gary Bruni 


Transportation Manager 


831-393-7922 


Gary brunifSictb.com 


Patricia Kinison 


Customer Service Project Lead 


317-205-0683 


Patricia kinison(S),ctb.com 


Donna Ilyland 


Delivery Manager 


303-721-4259 


dhyland(S),ctb.com 


Madalina Papacica 


Online Implementation Project Manager 


831-393-7276 


Madalina papacicafSl.ctb.com 


Jayne Snyder 


Business Requirements Analyst 


831-393-7452 


Jayne snvder(S).ctb.com 
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Program Description 



Scope 


Details 


Contract Duration 


2012-2014 


Fiscal Year Duration 


July 1 - June 30 


Grades 


j iniougn o 


Content Areas 


E/LA and Math for grades 3 through 8 
Science for Grades 4 & 6 only 
Social Studies for Grades 5 & 7 only 


Test 


Applied Skills (open-ended items) 

Multiple-Choice (selected-response items) 

CCSS Transition Pilot (open-ended and selected-response items) 


Testing Modes 


Paper and Online (mixed) 






Scope Items Covered 


Test Document and Materials Development 




Test Document and Materials Production 




Online Content Production 




Ancillary Materials Development and Production 




Document and Materials Manufacturing 




Enrollment, Fulfillment, and Delivery 




Assessment Design and Analysis 




Cutscore Setting for Science (Spring 2013 only) 




Web-enabled Services 




Online Assessment System 




Software Application Customization 




Document Processing 




Report Generation and Delivery 




Online Reporting 




Data Re-verification 




CTB/Indiana Help Desk 




MBE/WBE Information 






Enrollments (Public and 
Charter) 


-80,000 students per grade (total of 480,000) 




-1,600 schools (60 charter schools) 




-295 corporations 






Enrollments (Nonpublic 
and Choice schools) 


-5,000 students per grade (total of 30,000) 




-280 nonpublic and Choice schools 



Customer Participation 

- The IDOE will ensure prompt access to and decisions by the staff of the IDOE, by CTB or subcontractors 
as designated whenever such access and decisions are required in connection with the orderly progress of 
work for this program. 

The IDOE will participate in review and approval of the meetings outlined in the next section. 
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Customer Meetings 

CTB to provide the following: 
Meeting Agendas 

Schedule meetings and arrange for meeting facilities 

Pay meeting facility fees 

Prepare meeting materials 

Lead the meetings with IDOE staff 

Prepare and maintain documentation of procedures used/decisions made 
Provide meeting refreshments and meals 
Provide airfare for the IDOE staff 

Provide a per diem for IDOE staff traveling to Monterey, CA 
Provide a rental car for IDOE staff traveling to Monterey, CA 

- Provide lodging for IDOE staff in Monterey, CA 

Meetings for each administration: 
Planning Meetings 
Post-Test Meetings 
Status Meetings 
Test Deck Reviews 

- Test Coordinator Training Materials Review Meeting for ISTEP+ Online 
User Group meetings for ISTEP+ Online 

Science Cutscore Setting (Spring 2013 only) 

- Passage Review meeting (for Spring 2013 Applied Skills pilot administration only) 

- Content/Bias Review meeting (for Spring 2013 Applied Skills pilot administration only) 

- Anchor Review meeting (for Spring 201 3 Applied Skills pilot administration only) 

- Item review/alignment and selection meeting (for Spring 2013 CCSS Transition Online pilot 
administration only) 

- Anchor Review meeting (for Spring 201 3 CCSS Transition Online pilot administration only) 
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Program Management 

Negotiate all contract changes in scope, regardless of cost; document any changes in cost, schedule, or 
required deliverable. Ensure that customer and PM sign the change form (DCF) for any changes in 
schedule or scope. 

Follow up with customer and manage departments to ensure that the base-lined schedule is achieved, and 
that all requested changes in scope are managed so that the ultimate delivery dates for test and reports are 
met. 

Identify risks, rank, and mitigate all process risks that can impact the program schedule or the quality of 
deliverables. 

Develop the work schedule and process plans with assigned team members. 

Manage the program schedule to ensure that high quality deliverables are provided to the customer on 
time. 

Serve as point of contact between the IDOE and CTB departments. 
Submit Weekly Status Report and Action Item List to the IDOE. 
Facilitate weekly team meetings. 

Provide opportunity for timely communication between departments, and keep departments apprised of 
project status. 

Prepare and send correspondence to meeting attendees. 
Prepare customer meeting agendas for review by the IDOE. 
Schedule and conduct Red Team Reviews. 
Schedule and conduct with the IDOE the Test Deck Review. 
- Arrange IDOE visits to CTB . 
Prepare and submit invoices. 

Manage Change Orders and Decision Logs for the contract. 
Produce the Quarterly Error Report. 

Monitor Minority and Women Owned Business Enterprises (MBE/ WBE) contract compliance. 

Minority and Women Owned Business Enterprises Contract Compliance 

CTB agrees to comply fully with the provisions of 25 IAC 5 and the Subcontractor Commitment submitted 
with the State. No changes may be made to the commitment without the written approval of the Minority and 
Women's Enterprises Division of EDOA. 

The following MBEs and WBEs are listed on the Minority and Women's Business Enterprises Division 
directory of certified firms and will be participating in this Contract. 



Table 4 - M/WBE Vendors 



MBE 

or 
WBE? 


Telephone 
No. 


Company Name 


Scope of 
Product/Service 


Utilization Dates 


WBE 


317-578-7225 


All Pack Specialists, 
Inc. 


Provide corrugated 
boxes. 


December 2012 - May 
2014 


WBE 


260-434-0990 


Briljent LLC 


Training services 
including technology 
services, on site 
training and site 
survey. 


September 2012- 
May2014 


WBE 


856-486-0066 


Interactive Printing 
and Packaging 
Corporation (IPAK) 


Printing large print 
books, manipulatives, 
paper bands, stack 


December 2012 - May 
2014 
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cards and other 
materials. 




WRF 


317-872-4490 




TVansnortation and 
logistics 


January 2013- May 
2014 


MBE 


214-550-6124 


Dploylt 


Professional Staffing 
for contract 
employees 


January 2013 - August 
2014 


MBE 




Solo Printing 


Provide printed 
ancillary test 
materials 


January 2013 - April 
2014 



NOTE: The parties hereto understand that the subcontractors noted in the above table are used in an 
effort to reach 10% of the contract price. 

CTB agrees to submit a copy of the agreement entered into between CTB and each MBE/WBE subcontractor 
where the State took the selection of the MBE/WBE by CTB into consideration when issuing the procurement 
award. The copy of the agreement must be submitted to the MWBE Division in IDOA within ninety (90) 
days of the execution of the contract between CTB and the State. CTB also agrees to send all amendments, 
changes, and terminations to these agreements to the MWBE Division in IDOA within ninety (90) days of 
their execution. Failure to provide a copy of the agreement or subsequent amendment, change, and 
termination may result in exclusion from future State procurements. If CTB is not excluded from future 
procurements, the actions or inactions of CTB with regard to the above will be taken into account in all 
phases and scoring in future procurements. In addition, CTB must obtain the approval of the Division before 
changing any MBE/WBE participation plan submitted in connection with this Contract. 

Program Scope 



Program Milestones and Critical Dates 

Spring 2013 - See Attachment A 

- Spring 2014 - See Attachment A 

Out of Scope Work 

The following items are not included in the price or scope of this contract. If desired, they can be added for 
an increased cost. 

Spanish translations of reports 

Any report redesign work initiated at the request of the IDOE; as it is unclear how many reports will 
change, costs cannot be estimated for the new Operational test reports at this time 

- Beyond what is included in scope for the Spring 2013 and Spring 2014 pilot testing, any additional 
production of new items, the piloting of those items, and the associated workshops needed to approve the 
new items are not covered 

- Bulk/Group printing of online reports in INORS, with the exception of bulk print of IC Letters, ISRs and 
Image Print documents 

- Additional printing, fulfillment, shipping, or scoring of test materials over the amount stated in the Work 
Scope (see Document and Material Manufacturing) 

Teachers added to the Rangefinding meetings 

Changes to percentage of students taking online and paper/pencil test beyond what is included in scope 
for Spring 2013 and Spring 2014 administrations 

- Guide to Test Interpretation documents will not be printed. These will only be developed for online 
access in PDF form. 

- Additional printing of Grades 3-4 Practice tests (both Applied Skills and Multiple-Choice) and Grades 
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6-8 Gridded-Response Practice Tests (printing for one grade only is in scope) 

- Educator facing Teacher Scoring Guides will not be developed and delivered. 

Content Development/Publishing Deliverables 

The content development team assigned to the ISTEP+ program will work closely with the IDOE to build 
quality assessments that meet Indiana Standards and that provide valid, reliable results upon which 
educational decisions can be made. 

- Operational and Pilot Test Requirements and Test Design Tables are provided in Attachment E 

- CCSS Transition Online Pilot Test Requirements and Test Design Tables are provided in Attachment F 

Document and Materials Development 

Prepare Content Development specifications 

Develop test design, assessment limits 

Identify passages and items for department review 

- Conduct passage review and content/bias review meetings (for Spring 2013 Applied Skills pilot only) 
Art development and production 

- Conduct specifications and alignment meetings (for Spring 201 3 CCSS Transition Online pilot only) 

- Produce items, passages, tasks 
Produce field test forms 
Produce operational test forms 
Produce manuals and ancillary materials 

Provide updated PEED tables to Technology and Research 

Obtain copyright lines 

Obtain permissions 

Create custom cover designs 

Document and Materials Production 

Receipt of manuscript by Art and Production 

Creation of first pages 

Creation of second pages 

Customer review and approval 

Materials Integration Review (MIR) 

MIR administration 

Review and documentation of results 

Creation of third pages 

Query resolution 

Corrections made and reviewed 

- Pages submitted for prepress and QC 

Flight Check, postscript files, and convert to PDF 
Review of prepress output 
Release to Manufacturing 

- PEID files and item objective maps to Research and Technology 
Scoring guides and anchor lists to Research and Handscoring 

Online Content Production 

- Build content 

Tweak item formatting for online production 
Complete style reviews 
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Customer review and approval of online forms 
Corrections made and reviewed 
Deploy to CQA environment and verify 
Deploy to production 

- Produce and deploy to production Multiple-Choice Online practice tests for all grades 3-8 
Produce and deploy to production Applied Skills Online practice tests for all grades 3-8 
Produce and deploy to production Experience ISTEP+ Online to support both Applied Skills and 
Multiple-Choice testing experience 

Assessment Design and Analysis Deliverables - Research Tasks 

Develop pilot and validation sample school corporation list 

- QA of PEID files 

Create, QA and deliver pre-equated scoring tables for Operational and Braille tests to Technology (Final 
PAR Files for Operational and Braille tests to Statistical Services for scoring tables creation for all 
grades/content areas) 

- Develop and deliver IPI cut scores (Operational and Braille) to Technology 
Develop Analyses Specifications 

- Receive data for analyses in two waves: Applied Skills (Phase 1) and Applied Skills plus Multiple- 
Choice (Phase 2) 

- Use of standard Research Process Upgrade for: Classical Item Analyses, DIF, Calibration, and Equating 
for all operational items 

- Perform post-equating analysis with production slice data and prepare a report for IDOE 

Receive GRT from Scoring, create DAT files, run QA4IRT tool to perform an independent analysis and 

to validate IRT scoring 

Red Team review analyses of GRT only 

- Perform erasure analysis on the MC items and prepare a report for IDOE 

Complete Technical Report that covers analyses for Operational and pilot data and deliver to IDOE 
Create, QA, and Deliver Item Win DAT (or similar) files to Publishing 

- Form Selection training, specifications, and reviews 

Science pilot test and Cutscore Setting deliverables: Item analysis, DIF, calibration and scaling, linking (if 
possible) to current ISTEP Science, and full Cutscore Setting 

- Applied Skills Pilot (paper/pencil): Item analysis, DIF, and calibration and equating 

- Applied Skills Operational Online: Item analysis, DIF, post equating and comparability study of 
paper/pencil vs. online 

- CCSS Transition Online Pilot: Online MC and OE items only based on matrix design, Item analysis, DIF, 
calibration and scaling, linking (if possible) to current ISTEP+, and design/validate group-level scores 
only 

Comparability Study 

Pre-equating has been applied to ISTEP+ using 2009/2010 operational and subsequent field test item 
parameters. 

In 2009/2010, most students participated in paper-pencil administration mode, while a relatively small 
number of students participated in online administration mode. Further, the 2009/2010 item parameter 
estimates were based only on the responses from the paper-pencil administration mode. Over the years, there 
has been an increase in the number of students participating online, and the expectation is the numbers will 
continue to increase. As such, it is important to evaluate the stability of the item parameters estimated from 
only paper administrations with assessments for which online administrations are increasing. 

Two comparability studies are proposed and included in this scope: 

Page 14 of 63 <? u > 



Contract No. TBD 



(1) Pre-equating vs. post equating: Comparing the item parameters from the paper-only administration 
mode used for pre-equating to the item parameters of a post-equating based on the mix-mode (online and 
on paper) 2012 ISTEP population 

(2) Paper-pencil vs. online: Comparing the 2012 performance of students matched on ability across 
administration mode 

IDOE and CTB can then discuss the results of the studies to determine whether or not the item parameters 
currently used for the purpose of pre-equating require updating. 

Cutscore Setting Workshop (2013 only) 

3 -day Cutscore Setting workshop for Science grades 4 and 6 
Up to 24 educator participants 

Cut score recommendations to the State Board for approval 

- Updated performance level descriptions (PLDs - both complete and Student Report version) 

- Preliminary Cutscore Setting technical report, approximately 1 week after the workshop is completed 
Final Cutscore Setting technical report 



Test Design - General Information 




- There are 2 test administrations: one operational test form per grade for Applied Skills and 3 operational 
test forms per grade (1 original + 2 virtual ONLINE only forms ') for Multiple-Choice. 

- While the Applied Skills books and the grade 3 Multiple-Choice books are scannable, the grade 4-8 
Multiple-Choice books will be non-scannable with scannable answer documents 

- All grades will be testing E/LA and Math; Grades 4 and 6 will also be testing Science; and Grades 5 and 7 
will also be testing Social Studies 

Grade 3: E/LA, Math 
Grade 4: E/LA, Math, Science 
Grade 5: E/LA, Math, Social Studies 
Grade 6: E/LA, Math, Science 
Grade 7: E/LA, Math, Social Studies 
Grade 8: E/LA, Math 

Note: Test sessions will be administered in order: Math, then E/LA, then Social Studies (or Science). 
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- Spring 2013: About 10% of the total student population will take the Applied Skills test online through 
the OAS application for the two content areas E/LA and Social Studies, and the remaining 90% will be 
taking the paper/pencil tests. 

- Spring 2014: About 20% of the total student population will take the Applied Skills test online through 
the OAS application for the two content areas E/LA and Social Studies, and the remaining 80% will be 
taking the paper/pencil tests. 

- Applied Skills Mathematics and Science content areas will be administered 100% in paper/pencil mode 
only during both the spring 2013 and spring 2014 administrations. 

- Spring 20 1 3 : At least 65% of the total student population will take all content areas of the Multiple- 
Choice test online through the OAS application. Blocks of items will be moved to create two fixed 
virtual forms. 

- Spring 2014: At least 80% of the total student population will take all content areas of the Multiple- 
Choice test online through the OAS application. Blocks of items will be moved to create two fixed 
virtual forms. 

- Build a new online Applied Skills Practice test for each grade level (3-8) in 2013. Reuse these Practice 
Tests for subsequent administrations. 

- Reuse the existing online Multiple-Choice Practice Tests available or each grade level (3-8) in Spring 
2013 only. Additional Gridded-Response practice items are to be added to the online Practice Test for 
grades 6-8. 

- The Math Reference Sheet will be perforated in the Multiple-Choice operational books for Grades 6-8 
and printed as a separate sheet for Large Print books in Grades 6-8. The Math Reference Sheet is no 
longer perforated in the Applied Skills test books; it is a separate hand-out. 

- The Practice Tests for Grades 3-4 are 2 separate tests as follows: an Applied Skills Practice Test and a 
Multiple-Choice Practice Test, both of which will be posted online as well as printed for one grade only 
and shipped to sites. 

- There will be a Gridded-Response Practice Test for Grades 6-8, which will be posted online. The 
Gridded-Response Practice Test will be printed and shipped tor one grade only. 
Gridded-Response items for operational tests in Grades 6-8 

- Large Print for the operational tests at Grades 3-8 and ancillary materials to include Practice Tests, 
Reference Sheets, and Manipulatives 

- Braille for the operational test at Grades 3-8 and ancillary materials to include Practice Tests, Reference 
Sheets, and Manipulatives 

- CDs are available for Braille (VI), Large Print and IEP students (Large Print and IEP CDs are the same) 
in Grades 3, 4, and 5. The Braille and Large Print CDs will ship with the materials. 

Applied Skills pilot forms based on Indiana Academic Standards will be administered to a select 
population for all grades in ELA, Mathematics and Social Studies content areas. These pilot forms will be 
administered using paper/pencil and online modes. See Attachment E for Applied Skills pilot form 
design. 

- Multiple-Choice pilot forms based on CCSS will be administered to a select population for all grades in 
ELA and Mathematics content areas only, using the online mode. The Online Multiple-Choice pilot tests 
will assess and provide a transition to the state's readiness in implementing CCSS and will use CTB- 
owned CoreLink items. See Attachment F for this CCSS Transition Online pilot form design. 

Copyrights/Own ers hip 

- All work for ISTEP+ is custom 

- The IDOE owns all items, with the exception of CoreLink items used for the CCSS Transition pilot, 
which are owned by CTB. 

Manufacturing Deliverables 

Enrollment data 
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Operational and pilot test books (commodity codes, forms, and quantity by form) 

Test book colors agreed upon between the IDOE and CTB 

Test Coordinator's Manual 

Examiner's Manuals for each grade, 3-8 

Online Examiner's Manual 

Large Print Materials (operational and pilot test) 

Braille Materials (operational only) 

- IEP/LP and VT Audio CDs for grades 3, 4, and 5 (number of CDs to be determined, change from one per 
student to a ratio for each site) 

Print Samples 

Group Information Sheets* 

School/Group Lists* 

Test Coordinator's Kits 

Stack cover cards* 

Bands* 

- Posters* (two separate one-sided posters: one for Applied Skills and another for Multiple-Choice; both 
fulfilled during Applied Skills TCK materials in-site; 8 folds, different outer border color) 

* Items shipped in scoring kits 

Document and Materials Manufacturing 

Operational Test materials arc shrink-wrapped and packaged by 6's and by 12's, with manipulatives. For 
grades 6-8, Math Reference Sheets will be included in test book packages. All initial shipments will be 
sent in packages of 6 and 12. CTB will work to exhaust all packages of 12 in initial shipments. TMA 
shipments will be filled with packages of 6 and 12 (until 12's are exhausted). 
Grades 4-8 MC Answer books will be included in the test book packages, along with manipulatives. 

- Pilot test materials are spiraled by Class (or based on Research and IDOE decisions), shrink-wrapped and 
packaged by 12's (and 6's if needed). Consideration for mixed-mode piloting may be 
required/determined later. 

Cover Memos and ancillaries will be updated by Program Management and approved by IDOE prior to 
distribution. 

All the print quantities noted below in the tables are enrollment counts. Actual print quantities vary, 
based upon overage and packaging calculations. 



Table 5 - Printed Materials for Spring 2013 





Applied Skills Test 


Applied Skills Test 


Applied Skills Book 


Multiple-Choice 


Multiple-Choice 




Books (all content 


Books (Math and 


lis 


Test Books 


Answer Books 




areas) 


Science only) 










100% paper print 1 


10% paper print 


100% paper print 


35% paper 


35% paper 


Grade 3 


85,000 


8,500 


33,000 (1 1 forms) 


30,000 


NA 


Grade 4 


85,000 


8,500 


33,000 (1 1 forms) 


30,000 


30,000 


Grade 5 


85,000 


8,500 


42,000 (14 forms) 


30,000 


30,000 


Grade 6 


85,000 


8,500 


33,000 (1 1 forms) 


30,000 


30,000 


Grade 7 


85,000 


8,500 


42,000 (14 forms) 


30,000 


30,000 


Grade 8 


85,000 


8,500 


33,000(11 forms) 


30,000 


30,000 



1 Contingency in ycar-1 , printing 100% pnpcr materials for Applied Skills since readiness of student testing online Applied Skills is yet to he 
determined. 
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Table 6 - Printed Materials for Spring 2014 





Applied Skills Test 
Books (all content 
areas) 


Applied Skills Test 
Books (Math and 
Science only) 


Multiple-Choice 
Test Books 


Multiple-Choice 

a n l 

Answer Books 


Assumption 


80% paper print 2 


20% paper print 2 


20% paper 


20% paper 


Grade 3 


68,000 


17,000 


17,000 


NA 


Grade 4 


68,000 


17,000 


17,000 


17,000 


Grade 5 


68,000 


17,000 


17,000 


17,000 


Grade 6 


68,000 


17,000 


17,000 


17,000 


Grade 7 


68,000 


17,000 


17,000 


17,000 


Grade 8 


68,000 


17,000 


17,000 


17,000 



2 Based on readiness studies from 2013, an option to print 100% paper is included in the event online testing for Applied Skills is to be de-scoped. 



Table 7 - Braille, LP and Ancillaries for Spring 2013 



Item Description 


Applied 
SIdlls 


Applied 
Skills Book 
lis 


Multiple- 
Choice 


Comments 


Regular Population 










Grade 3-5 IEP/LP CDs 


25 


10 


25 


Approx.; based on 2012 


Grade 3 Manipulative 


85,000 


NA 


30,000 




Grade 5 Manipulative 


85,000 


NA 


30,000 




Grade 4, 6-8 Manipulative 


340,000 


NA 


120,000 


For 4 grades 


Grade 6-8 Reference Sheet 


255,000 


NA 


90,000 


For 3 grades 


Grade 3-4 Practice Test 


85,000 and 
PDF Online 


NA 


85,000 and 
PDF Online 


Printed only for one grade 


Grade 6-8 GR Items Practice Test 


NA 


NA 


85,000 and 
PDF Online 


Printed only for one grade 


Guide to Test Interpretation 


PDF via TNORS 


10% changes from year to 
year 


Test Coordinator's Manual 


5,000 


Ratio of 2 per corporation 
and 2 per school; 
exceptions noted in 
Table 9 


Grade 3 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 


Grade 4 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 


Grade 5 Examiner's Manual 


9,000 


Ratio of 1 EM for every 1 
students taking paper/pencil test 


Grade 6 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 


Grade 7 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 


Grade 8 Examiner's Manual 


9,000 


Ratio of 1 EM for every 1 
students taking paper/pencil test 


Applied Skills Online Examiner's 
Manual 


2,000 


All grades combined; ratio of 1 
EM for every 10 students taking 
AS online test 


Multiple-Choice Online Examiner's 
Manual 


45,000 


All grades combined; ratio of 1 
EM for every 1 students taking 
MC online test 


Stack Cover Cards 


Stack Cover Cards and Paper Bands are 
supplied at a ratio of 1 set of 12 cards and 24 
bands per 150 students at all grades. Additional 
sets of 24 bands only will be available through 
TMA. 




Paper Bands 
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Item Description 


Applied 
Skills 


Applied 
Skills Book 
lis 


Multiple- 
Choice 


Comments 


Posters 


2,500 


NA 


2,500 


Posters are supplied at a 

vfltin r\f 1 npr pninni'Atinti 

(ilUUUI I pwl ^Ul^JUl ULIUH 

finfl 1 np.r Qchnnl 

UUU 1 j-'Vi OvItUUli 

Additional posters will be 
available through TMA. 


SGL 


3 per school 




GIS 


1 per 8 students 




Return Shinnins Isabels 


Corp level: 1 plus 1 per 1000 students 
School level; 1 plus 1 per 50 - 75 students 




Cartons 






Braille Materials . 










Grade 3-5 VI CDs 


60 


NA 


60 




Grade 3 Manipulative 


20 


NA 


20 




Grade 5 Manipulative 


20 


NA 


20 




Grade 4, 6-8 Manipulative 


80 


NA 


80 




Grade 6-8 Reference Sheet 


60 


NA 


60 




Grade 3-4 Practice Test 


30 


NA 


30 




Grade 6-8 GR Items Practice Test 


NA 


NA 


60 




Large Print Materials 










Grade 3 Manipulative 


100 


NA 


100 




Grade 5 Manipulative 


100 


NA 


100 




Grade 4, 6-8 Manipulative 


400 


NA 


400 




Grade 6-8 Reference Sheet 


300 


NA 


300 




Grade 3-4 Practice Test 


100 


NA 


100 




Grade 6-8 GR Items Practice Test 


NA 


NA 


300 




Table 8 - Braille, LP and Ancillaries for Swing 2014 




Item Description 


Applied 
Skills 


Applied 
Skills Book 
lis 


Multiple- 
Choice 


Comments 












Grade 3-5 IEP/LP CDs 


25 


10 


25 


Approx.; based on 2012 


Grade 3 Manipulative 


68,000 


NA 


17,000 




Grade 5 Manipulative 


68,000 


NA 


17,000 




Grade 4, 6-8 Manipulative 


275,000 


NA 


68,000 


For 4 grades 


Grade 6-8 Reference Sheet 


205,000 


NA 


51,000 


For 3 grades 


Grade 3-4 Practice Test 


85,000 and 
PDF Online 


NA 


85,000 and 
PDF Online 


Printed for one grade only 


Grade 6-8 GR Items Practice Test 


NA 


NA 


85,000 and 
PDF Online 


Printed for one grade only 


Guide to Test Interpretation 


PDF via INORS 


10% changes from year to 
year 


Test Coordinator's Manual 


5,000 


Ratio of 2 per corporation 
and 2 per school; 
exceptions noted in 
Table 9 


Grade 3 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 


Grade 4 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 


Grade 5 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 
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Item Description 


Applied 
Skills 


Applied 
Skills Book 
Us 


Multiple- 
Choice 


Comments 


Grade 6 Examiner's Manual 


9,000 


Ratio of 1 EM for every 1 
students taking paper/pencil test 


VJictutJ / C/Adlllillci o lvicuiudl 


9,000 


Ratio of 1 EM for every 1 
students taking paper/pencil test 


Grade 8 Examiner's Manual 


9,000 


Ratio of 1 EM for every 10 
students taking paper/pencil test 


Applied Skills Online Examiner's 
Manual 


4,000 


All grades combined; ratio of 1 
EM for every 10 students taking 
AS online test 


Multiple-Choice Online Examiner's 
Manual 


53,000 


All grades combined; ratio of 1 
EM for every 10 students taking 
MC online test)\ 


Stack Cover Cards 


Stack Cover Cards and Paper Bands are 
supplied at a ratio of 1 set of 12 cards and 24 
bands per 150 students at all grades. Additional 
sets of 24 bands only will be available through 
TMA. 




Paper Bands 




Posters 


2,500 


NA 


2,500 


Posters are supplied at a 
ratio of 1 per corporation 
and 1 per school. 
Additional posters will be 
available through TMA. 


SGL 


3 per school 




GIS 


1 per 8 students 




Return Shipping Labels 


Coip level: 1 plus 1 per 1000 students 
School level: 1 plus 1 per 50 - 75 students 




Cartons 






Braille Materials i 










Grade 3-5 VI CDs 


60 


NA 


60 




Grade 3 Manipulative 


20 


NA 


20 




Grade 5 Manipulative 


20 


NA 


20 




Grade 4, 6-8 Manipulative 


80 


NA 


80 




Grade 6-8 Reference Sheet 


60 


NA 


60 




Grade 3-4 Practice Test 


30 


NA 


30 




Grade 6-8 GR Items Practice Test 


NA 


NA 


60 




Large Print Materials 










Grade 3 Manipulative 


100 


NA 


100 




Grade 5 Manipulative 


100 


NA 


100 




Grade 4, 6-8 Manipulative 


400 


NA 


400 




Grade 6-8 Reference Sheet 


300 


NA 


300 




Grade 3-4 Practice Test 


100 


NA 


100 




Grade 6-8 GR Items Practice Test 


NA 


NA 


300 





Table 9 - Test Coordinator's Manual Quantities for Large Corporations 



NAME 


Corptf 


# of TCMs 


NAME 


Corp# 


ft of TCMs 


INDIANAPOLIS PUBLIC SCHOOLS 


5385 


50 


CLARK-PLEASANT COM SCHOOL CO 


4145 


5 


FORT WAYNE COMMUNITY SCHOOLS 


0235 


20 


CENTER GROVE COM SCH CORP 


4205 


5 


HAMILTON SOUTHEASTERN SCHOOL 


3005 


20 


WARSAW COMMUNITY SCHOOLS 


4415 


5 


SOUTH BEND COMMUNITY SCH COR 


7205 


20 


MERR1LLVILLE COMMUNITY SCHOO 


4601) 


5 


EVANS VILLE VANDERBURGH SCH C 


7995 


20 


LAKE CENTRAL SCHOOL CORP 


4615 


5 


DISTRICT - INDIANAPOLIS 


9200 


20 


CROWN POINT COMMUNITY SCH CO 


4660 


S 


GREATER CLARK COUNTY SCHOOLS 


1010 


10 


GARY COMMUNITY SCHOOL CORP 


4690 


5 


ELKHART COMMUNITY SCHOOLS 


2305 


10 


MICHIGAN CITY AREA SCHOOLS 


4925 


5 
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NAME 


Corptf 


flofTCMs 


NAME 


Corp// 


ffofTCMs 


NEW ALBANY-FLOYD CO CON SCH 


2400 


10 


LAPORTE COMMUNITY SCHOOL COR 


4945 


S 


LAKMbLCLAY SCHOOLS 


JUOU 


1 A 


ANDERSON COMMUNITY SCHOOL 

AA\ 




-j 


oL.MUUL<C1J Y Ui MAJVlIVlUiNlJ 


HI 1U 


in 

1U 


ucn IYPPATI IR TOWNSHIP 


5300 


5 


lipni AXl/DCMA'E TA\\XfMCl-JTD 

M o IJ LAWKbNOb lUWNoriir 




in 

ID 


FR ANIf I FN TOWNSHIP POM SPH PO 


5310 


5 


\A O r\ T1CI>T>V TVMl/MCLITD 

M a U PbKKY IUWNdHIH 




in 


luT Q r> WACTJTMfiTfYM THVi/MSMlP 
IY1 L> W/\OrlHNU 1 UiN 1 YV IN.ZH 1 1 1 


5370 


5 


ivl o U ritvi; luwrNoiiii 


ST;n 


in 


MONROF COI INTY COM SCH CORP 


5740 


5 


\A C r\ \X/ A DD DM TAWUKTCI4ID 

ivl o IJ WAKKblN lUWNonlr 


JJOU 


in 


M S n MARTINSVir I F SOHOOI S 


5925 


5 


MaU WAYNb lUWNolilr 


JJ IJ 


in 






s 


TIDDCPAWHC C A* 14 A^ All PADD 


/oO_> 


in 


PORTAfiF TOWNSHIP SCHOOT S 


6550 


5 


\nnr~\ av~ii rniTV c^wnfM padp 

VUjvJ \_-UUIN J i aOnUUii l^VJIVr 


OUJV 


in 


VAI PAR AISO COMMUNITY SCHOOI S 


6560 


5 


Ul J 1 KJS- I'll W oU DtlNU 


Q77fl 


in 


PFMM-HARRIS-MADISON SCH CORP 

I CI ' 1 i 1 1 r\ lij IVl/V t-J 1«J V_/l » QUI 1 V»V/1 \ 1 


7175 


5 


MbD oUU 1 MWbo I ALLbN CUUM I Y 




c 




I AFAVRTTR <;PHnni POT?POR ATIDN 

LAPA IDJ JC O^I l^JUL/ LUIVI \Ji\s\ 1 i\Jvi 


7855 


5 


WADTmi/CQT AIT CM A7~\l IKITV CPHAO 




e 

M 


WAR R \CK COI INT Y .SrHDOI , CORP 


8130 


5 


HACT AT T n\1 AAI IMTV Q At-TAAMT Q 
UAol ALLcN L.UU1N1 I oL-rlUULo 




C 




8888 


5 


D A D TT-TA\T A\K/fn\X/ A*A»Xt CA*HA1AM A7~\PP 


(HAS 


c 


ni*\TR TPT — OAR Y 


9205 


5 




UOJU 


c 

3 


niCTRIPT — RVAN^VIT I F 
JJIO 11x11*1 — li V/ilNO V IJ-'I-.CV 


9215 


5 


KAI rhlATC ^T»K A\ A} TMITV COTJOOT C 

MUNLlb CUMMUNli Y bCiiUULo 




C 


DKTR 1PT — IT 1THPR AN 


9230 


5 


fiOSHFN COMMUNITY SCHOOI S 


2315 


5 








WESTFIELD-WASHINGTON SCHOOLS 


3030 


5 








NOBLESVILLE SCHOOLS 


3070 


5 








BROWNSBURG COMMUNITY SCH COR 


3305 


5 








AVON COMMUNITY SCHOOL CORP 


3315 


S 








KOKOMO-CENTER TWP CON SCH CO 


3500 


5 








HUNTINGTON CO COM SCH CORP 


3625 


5 









Enrollment, Fulfillment, and Delivery 



- Prepare Fultlllment/Traffic Specifications 
Assist with the Enrollment process 

- Fulfillment will receive an enrollment file from the IDOE with student enrollments by corporation, 
school, and grade. 

- The Enrollment File will need validation from the IDOE regarding updates and discrepancies. 

- In addition to the Enrollment File, the IDOE needs to provide a list of reconfigured, closed, newly 
opened, and renamed schools. 

- Any changes to the Enrollment File by the IDOE after the final Variance Review could cause delay in 
material deliveries. These late deliveries are not subject to liquidated damages. 

Use Research-provided pilot site selection for fulfillment and delivery of pilot materials 
Materials packaged by corporation, school, nonpublic school, charter school, and choice school 

- Order Forms for enrollment collection for Large Print, Braille, and audio CD's can be found on the 
<www.ctb.com/istep > webpage for online ordering through the CTB Help Desk. 1 

- Enrollment contacts at IDOE are Michele Walker and Charity Flores. 
Packing Lists are to be used with all shipments. 

1 Ordering process will be revisited with IDOE for 2013 and 2014 administration years 



Web-Enabled Services Deliverables 

Portal Page (www.ctb.com/istcp) 

Unsecured program portal page for educator access 




Contract No. TBD 

Portal customized for organizing materials by user-group types (Test Coordinators, Tech Coordinators, 
etc). 

- Single point of access for all ISTEP+ related communication and support materials to educators 



ISTEP+ Online Forum 

Secured web-based community forum for educators 
Educators register and create accounts themselves. 
- CTB maintains and moderates the community forum; posts materials and messages, as needed for the 
ISTEP+ administration. 



Experience ISTEP+ Online 

- Unsecured web-based instructional and practice tool for ISTEP+ Online 
Will mimic all online tools available for students during operational testing 

- Tool to provide instructional and practice opportunities for both Applied Skills and Multiple-Choice tests, 
using items from the Practice Test and released Indiana item pool. 

Single Sign-on (SSO) User Accounts 

- Generate a single user-account for each of the State, Corporation, and School Coordinators/users to access 
all web-enabled applications 

- Enable access to all educator facing web applications - ctb.com, Acuity, Indiana Online Reporting System 
(INORS), and Online Assessment System (ISTEP+ Online or OAS) - with the use of these SSO accounts 
Exceptions: 

o Teacher accounts are not in scope for SSO account creation. Individual teacher accounts for 
access to Acuity and INORS will continue to be created and maintained outside of SSO 
credentials. 

o SSO accounts do not apply for the Parent/Network (PN). PN will have accounts created for 

parents and educators outside of the SSO user database, 
o SSO accounts do not apply for the ISTEP+ Online Forum. Educators can request and gain access 

to the ISTEP+ Online forum, and these accounts are outside of the SSO user database. 

- CTB/Indiana Help desk can issue a new username/password. 

Online Short/Adds (TMA) 

Secured application, entry will be permitted to test coordinators only with their SSO accounts 

- Two TMA administration windows for each administration year will be created and supported, one for 
Applied Skills and another for Multiple-Choice. 

Cover memos, user guides and recorded webinars will be provided by CTB. 
The customer will determine the messages posted in the message box. 

- TMA must include the functionality of ordering Braille and Large Print materials for nonpublic, charter, 
and choice schools. 

Secure File Transfer 

- For educators: Secured access using SSO accounts to Secure File Sharing Location (ISOFT) 

- For secured data/document handoff between IDOE and CTB, a secured file sharing application tool will 
be made available by CTB, along with user-accounts for IDOE staff. 
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Shipping Service 

Shipping Mode: Freight Carrier or UPS Ground service based on shipment size for efficiency (unless 
contract delivery in-site date cannot be met) for the material deliveries to corporations, nonpublic schools, 
charter schools, and choice schools 

- CTB Navigator-Online scheduling system (TACS) will NOT be used. 

- Return Label Color coding will be discussed with IDOE and agreed upon per recommendations from the 
Scoring Team. 

- Pilot and Validation school corporations will be picked up early for both Applied Skills and Multiple-Choice 
test administrations. 

Langham Logistics is the carrier for pick-up of ALL materials (exception: sites calling). 

- Langham Logistics delivers Applied Skills and Multiple-Choice materials to most corps (criteria is 
corporation size and/or # of boxes), and UPS delivers the rest. 

- All TCKs, OEMs and score reports are shipped via UPS. 

Online Assessment System (OAS) 

See Attachment B for detailed OAS scope information 

- All product fiinctionalities/tools implemented in Spring 20 12 will be supported. 

- Additional changes based on user-feedback received will be reviewed with IDOE and scope changes will be 
reviewed through change control process. 

Software Application Customization 

- Prepare Technology Specifications, scoring and reporting business rules, and mockups, including data file 
layouts 

- Develop image program for scanning new test forms and handoff image PDF files to online reporting 
systems 

- Develop Pre-Id data load and barcode label print programs for both Applied Skills and Multiple-Choice 
tests 

- Develop Edit/Raw Score programs for operational and pilot test forms 
Develop Handscoring programs for scoring the Applied Skills tests 

Develop Scoring and Reporting programs for new test forms, including functionality related to the match- 
merge for two Applied Skills books and then with the Multiple-Choice tests 

- Develop reporting programs to split report Science content area from the rest during Spring 20 1 3 due to 
Cutscore setting activity for Science 

- Develop biographic (RES) update and unmatch resolution update programs to update STN and 
demographic updates received from IDOE 

Develop pre-id extract functionalities to online and reporting sub-systems 

- Load and QA final scoring tables and items contributing to final operational scale score 

- Prepare for and attend Red Team Review 
Prepare for and lead Test Deck review with IDOE 

- Work with the Indiana Online Reporting System (INORS) and Parent Network (PN) components and 
provide all data handoffs necessaiy to enable those systems to report ISTEP+ data 

- Develop and setup EHS rescore functionality and custom EHS reports 

Scoring Deliverables 

Prework Requirements 

Prework required 

Precode Group Information Sheet (GIS) 

- School/Group List (SGL) 
Return Box Labels 
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- Student Barcode Labels and Student Rosters Ki Yes dNo 
Precode Barcode 

Specific information regarding barcodes will be provided in the Technology Specifications. 

- Barcodes will be provided for both the 2013 and 2014 test administrations in 2 phases. Edit software will 
be used. The barcodes are sorted using the "file in-file out" method. One copy of the roster is sent to the 
schools. CTB will use the label design that has been previously approved by the IDOE. 

- A sample barcode file will be sent from the IDOE to Scoring approximately 1 days before the scheduled 
delivery date of the Phase 1 barcode file for a QA check to make sure the format of the data is correct. 
Barcode Phases: 

- Phase 1 - All Applied Skills student population (mid-January) 
Phase 2 - New/moved students for Multiple-Choice (mid-March) 

- CTB will run Frequency Distribution on both Pre-ED Phase 1 & 2 files, and results will be provided to TDOE 
for review and approval prior to generating barcode label print. 

Print one set of labels for all students in Phase 1 

- Print one additional set of labels for all students combined (phase 1 and phase 2) to place in MC TCK 
Shipment will be sent to the corporation (and nonpublic schools) prior to each administration and will 
include 2 labels for each student. 

Ancillary Systems 

There are multiple master hierarchy files used (public school corporations, nonpublic schools, charter 
schools, and choice schools) for the ISTEP+ program. The files will contain the following information: 

Corporation number and school number 

- Flag indicating that all schools are public (any private schools) 
Flag indicating participating schools 

Corporation name and address (short names) 
Enrollment per grade by site 
School name and address (short names) 
Contact person name at the school 
Phone number of contact person 

Document Processing 

Scoring will prepare detailed specifications for all phases of Production Processing, completeness checks, 
data verifications, and Reports Generation. 

Prework: Fulfillment will deliver the Master Hierarchy file number to Scoring. Scoring will load and 
review the Master Hierarchy file and compare to previous year's data and report discrepancies and any 
variances noted to Program Management. 

Scoring will set up Organization testing programs and build a hierarchy for processing, scoring and 
reporting. 

- Scoring will lock down Mainframe Prework: Tracking records , school/group list application, stack 
headers and prework print on demand for document processing prior to the Applied Skills receiving 
window 

During pre-processing window, 2 sets of Test Coordinator Kits will be delivered: one for the Applied 
Skills delivered prior to testing window, and a separate set for Multiple Choice shipped prior to the 
Multiple-Choice testing window. Scoring Test Coordinator Kits will include TCK Cover memo, SGL, 
G1S, Return box labels, Student Barcode labels, Barcode Rosters, Test Coordinator's Manuals, Stack 
cover cards, Bands and Posters. No TCM or Posters will be included in the Multiple-Choice TCKs. 

- IDOE will provide 1 sample file and two phases of Student Barcode files to CTB. A Frequency 
distribution report will be run on each phase with output provided to the IDOE. Phase 1 files will be 
printed and delivered with Applied Skills TCK #1 . The phase 2 barcode file will only consist of new or 
move-in students who were not included in the first file. A complete set of all students' barcodes labels 
from Phases 1 and 2 will be generated and reprinted to be delivered with the Multiple-Choice TCK #2. 

Page 24 of 63 ^-rfy 

o 



Contract No. TBD 



Scoring will provide Technology with Student Master File after each phase for mainframe deployment 
prior to the Applied Skills scanning and again prior to Multiple-Choice production. 
Provide OAS group with extract files for corps/schools testing online. (Used to set up hierarchy for 
Multiple-Choice online testing) 

User Acceptance Testing : Internal CTB Applied Skills scoring test deck processed through DPC and 
Handscoring for user acceptance testing to ensure front-end and backend software applications are 
functioning as intended. Internal Multiple-Choice test deck processed through DPC for user acceptance 
testing, as well. 

IDOE test deck through front-end, after final Production software is released 

Production Processing : Unique op units will be assigned for the processing of the Validation sample, 
Pilot Sample, Operational and Multiple Choice data coming from OAS. 
Applied Skills : 

Applied Skills Pilot Book II data will be processed on a separate track and Research will obtain data 
through the RPU process. 

Upon retrieval of Applied Skills, material will be received and logged in, images scanned and data edited 
through WinScore and items hand-scored through EHS system. 
After export, bio data will bridge to the mainframe. 
Scoring will verify completeness using the Monarch system. 

The match merge process will be triggered by the bridging of Applied Skills data for students taking 
certain parts online and other parts paper/pencil. 

After Applied Skills data are complete through the Handscoring system, scored CR items will be exported 

from EHS and Op units FTP'd to the mainframe and Scores loaded to EISS. 

Image Print Rollup : Development will work with Technology to pre-identify image print page 

specifications. 

Scoring will verify corporation completeness for image print roll-up and will report problematic issues to 
Technology for clean-up. After each corporation is completed through Scanning and all their op units 
export from WinScore, student images go through book rollup process. A validation check is performed 
against the number of books exported from WinScore as compared to the number of books in the book 
rollup summary report. Images automatically FTP to holding site using a paperless image print solution 
for the INORS and Parent Network systems. 

STN and Demographic updates : Provide IDOE with Extract file for STN and demographic updates for 
resolution 

Provide Applied Skills Duplicate File to IDOE for resolution 
Provide IDOE with List of Late Cases received after cutoff date 

Bio Extract files are delivered to Parent Network for generation of Invitation Code letters. 
Data Completion : All discrepant Op units are reviewed and researched. Customer is sent email 
notifications of all unresolved cases count discrepancies. A stand alone match merge is run after final 
data clean-up prior to Multiple-Choice processing. 
Multiple-Choice Production Processing : 

CCSS Transition Online pilot data from online systems will also be processed on a separate track, and 

Research will obtain data through the RPU process and match data with Applied Skills. 

Upon retrieval of Multiple-Choice, answer documents will be received, logged in, scanned, and scored 

and data edited through WinScore and exported to the Mainframe. 

Multiple-Choice Validation Sample will be prioritized for Research. 

The match merge process will be triggered by the bridging of Multiple-Choice data. 

After Applied Skills and Multiple-Choice clean-up/match merge, Scoring will deliver Matched Validation 

data to Research in GRT format using final tables. 

After all Operational scoring and match merge is complete, a final GRF and reports for Red Team Review 
will be provided to Technology and Research. 

Participate in Red Team Review. After Red Team is approved by IDOE and final sign-off is received, 
reports are produced and released in 5 phases (see reports generation section for details), 
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- Unscannable Image Print: Unscannable documents (to include Applied Skills Large Print books and 
computer-generated responses) are printed and shipped to corporations/schools for review prior to the 
rescore window. 

Handscoring: Operational scoring 

- For the Operational test in Grades 3-8, the 5% sample for 2nd reads is for research purposes and will be 
read to resolution. 

- Handscoring will not be prioritizing a Validation sample during the operational scoring window. 
Condition Codes will be scored by evaluators with approval from an expeit evaluator: 

A = Blank/no response 

B = Illegible/unable to score 

C = Written predominantly in language other than English/unable to score 
D = Insufficient response/unable to score/copies from text 
E = Response not related to test question or scoring rule 
NOTE: All condition codes convert to points obtained 
Condition Codes 

Multiple-part items cannot receive a condition code and a score with the exception of condition code E 
(Off topic/Response not related to test question or scoring rule for E/LA, Writing, Social Studies and 
Science only). Social Studies condition codes will be the same as ELA and science condition codes, 
i.e., A-E. For ELA, a student can receive condition code E for WA and a score for LC. 
- Applicable condition codes for Math will be A-D. All math items will receive 2 scores. Math items 
cannot receive different condition codes for each part of the same item. 

- If an evaluator is released for failing to maintain standards, the IDOE and CTB will discuss whether 
his/her reads for a particular RIB will be reset back to the time at which the evaluator fell below standards 
or reset back to the beginning. 

- If a student completely erased his/her response, a condition code A will be assigned. If there is an 
incomplete erasure, the evaluator will assign a score based upon the non-erased section. 

Image Printing Systems are used. 

- 1 -item RIBS 

- All CR items will be scored using a holistic rubric except for Science and Social Studies. 

- All scoring will be done at the CTB site in Indianapolis. (Any changes to the site will be communicated to 
and approved by the IDOE.) 

Handscoring: Pilot scoring 

The Handscoring team will facilitate an Anchor Review meeting with IDOE and develop training 
materials for scoring pilot forms. 

- The Handscoring team will handscore pilot data after IDOE's review and approval of the TMRM CD. 

- New anchor/training materials will be prepared for pilot items that were edited. This is necessary because 
of changes in what is being asked of the students. 

Rescore Process 

- The INORS team will load students into the 1NORS system. 
Images will be available in the INORS system. 

Undetermined Student Reports will be online. Parents must initiate the rescore request process. Teacher 
will submit request. Rescore requests will be made on line, 

- The INORS team will provide the rescore file from the INORS system, which will be uploaded to EHS. 
Experienced Handscoring Evaluators will review training papers and take a qualification round. After 
successfully qualifying, evaluators will perform a blind read of the items to be rescored. After completing 
the blind read, any scoring disagreements between the students' reported score and the blind rescore will be 
identified and submitted to the content area Supervisor(s) for review. 

- The content area Super visor(s) will conduct an in-depth analysis of all the items in which there is a scoring 
disagreement. After reviewing and analyzing student rcsponse(s), the content area Supervisor(s) will assign 
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final scores to all contested items. If this score differs from the student's reported score, the Supervisor(s) 
will enter a new score and document the reason for the error and assign one of the eight (8) codes for rescore. 

- Rescore data will be exported from EHS and scores loaded and QA'd. 

- Report of Rescoring will be Red-Teamed and approved for print. 

- Scoring will generate and print the Report of Rescoring for all students whose scores have been changed 
during the rescore period. Only students with score changes will receive new Student Reports and Student 
labels. 

- Post rescore files will be delivered to the INORS and Parent Network teams. 

- Updated e-files to IDOE 

- Phase 5 cleanup and reporting window added to accommodate late issues of concern 
Report Generation and Delivery 

The reports and electronic file deliverables are as listed in the below Table 10. 

- Spring 2013: Due to Science Cutscore Setting, reports in 2013 will be split into two rounds: one for 
E/LA, Mathematics and Social Studies following the expedited reporting TAT; and the other, including 
Science scores, later in the summer. 

- Spring 20 14: Reports for all content areas will be reported together, following the expedited reporting 
TAT. 

- CCSS Transition Online test-based reports will not follow the expedited reporting TAT. These reports 
will be generated and released once after all of the Research analysis is completed in the summer, and the 
EDOE has approved the release of these reports to educators. 



Table 10 - Reports and Electronic Files Matrix 



Report Name 


Grd 


n> 


Mode 


Paper 


INORS 


INORS 


Parent 










Copy 


PDF 


Online 


Network 


Academic Standards 
Frequency Distribution 


All 


STFD 


St,P 






X 




Academic Standards 
Summary 


All 


ESSM 


St,P 






X 




Applied Skills Frequency 
Distribution 


All 


ASFD 


St, P, s 






X 




Class Academic 
Standards Report 


All 


CPR 


c 






X 




Class Proficiency 
Grouping Report 


All 


CPGR 


c 






X 




Disaggregation Summary 
Report 


All 


SDSA 
(DISA) 


St, P, s 






X 




Group Academic 
Standards Summary 


All 


GESS 


St,P 






X 




Invitation Code Letters 


All 


NA 


S 


X 


X 




X 


ISTEP+ Student Report 


AH 


ISR 


S 


X 








ISTEP+ Student Report 


All 


ISR 


c 




X 




X 


Image PDF of Student 
Responses 


All 


NA 


c 




X 




X 


Perfect Scores Report 


All 


PFSR 


St 


X 
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Grd 


ID 


Mode 


Paper 

Copy 


INORS 
PDF 


INORS 
Online 


Parent 
Network 


Proficiency Peiformance 

Ls U J 1 11 1 iCXl j 


All 


SPPR 


St, P, S 






X 




Proficiency Roster 


All 


PPR 


C,S 






X 




Report of Rescoring 


All 


NA 


p 










Rescore Request Form 


All 


RRF 


s 






X 




Rescore Request Log 


All 


NA 


St, P, s 






X 




Student Label 


All 


LBL 


s 


X 








Undetermined Status 
Roster 


All 


UDTR 


s 






X 





Legend: St = State, P = Corporation, S = School, C = Class 



File Name 


Grd 


ID 


Mode 


File 


Delivered 


Delivered 


Comments 










Type 


to 


Via 




Disaggregation File 
(and layout) 


All 


e-DISA 


St, P, 
S 


.dat 


State 


TeamSite 




General Research 
Tape/File (and layout) 
- corporation version 


All 


GRT/GRF 


S 


Comma 

delimited 

(.csv) 


Corp. 
School 


INORS 




General Research 
Tape/File (and layout) 
- state version 


All 


GRT/GRF 


c 


text 


State 


TeamSite 




Invalid STN and RES 
Update File 
(combined) 


All 


NA 


NA 


Excel 
(.xls) 


State 


TeamSite 




Invitation Codes File 
(and layout) 


All 


NA 


S 


Comma 

delimited 

(.csv) 


Corp 
School 


INORS 




Media File (and 
layout) 


All 


MEDR 


S 


.dat 


State 


TeamSite 




Summary File (and 
layout) 


All 


SUMT 


St,P, 
S 


.dat 


State 


TeamSite 




Unmatched Resolution 
File 


All 


NA 


NA 


Excel 
(.xls) 


State 


TeamSite 





Legend: St = State, P = Corporation, S = School, C = Class 



Table 11 - CCSS based Transition Reports 



Report Name 


Grd 


ID 


Mode 


Paper 
Copy 


INORS 
PDF 


INORS 
Online 


Parent 
Network 


Group Report 1 


All 




St 




X 






Group Report 2 


All 




P,S 




X 
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Legend: St = State, P = Corporation, S = School, C = Class 
Delivery of Score Reports 

CTB will deliver Phase 1 reports through INORS and Parent Network within five days of receipt of the final 
Multiple-Choice answer documents from the Indiana corporations to the Indianapolis Scoring Site. The five- 
day reporting schedule is dependent upon pre-equating of the ISTEP+ test forms. Prior to release of results, 
CTB will present IDOE with a longitudinal review of pre-equated results. Turnaround time for reporting is 
also dependent upon the state of documents and data received from corporations and schools and data clean- 
up protocols agreed-upon by the IDOE and CTB. Factors that will impact the five-day turnaround time 
include: 

- Less than 65% of students taking ISTEP+ Multiple-Choice online through OAS 
A decision by the IDOE to not use pre-equated results 

- Unanticipated test materials or data, outside of the control of CTB, that would impact processing 

The following are some factors that may impact the five-day turnaround for individuals or groups of students. 
Program Management will contact the IDOE within 12 hours if any documents received fail to meet CTB's 
basic processing standards and discuss a process of resolution agreeable to both parties. 

Barcode to Biographic discrepancies - Written name does not match affixed barcode label or incorrect 

label affixed to the student document 

No STN numbers or incorrectly code STNs for students with no barcode labels 

- Used wrong pencil type/student writing too light - Multiple-Choice responses would not be picked up 
Students tested in ink - creates undetermined situations which generate customer queries 

- Procedures not followed for handling, packaging, and return of non-standard documents - Large Print, 
computer-generated, and Braille - creates undetermined situations which generate customer queries: 

Non-standard documents are not listed on the SGL in the comment section. 

- NSR is not indicated in the response section of the Applied Skills books. 
YES bubble is not bubbled for computer-generated responses. 

Pastel color not used for computer-generated responses - removes ease of detection 

Braille students returned with regular population instead of separately or partial material sent back as 

Braille - on search mission to locate documents, which introduces duplicate records 

Late return of Braille - reverse translation can be affected 

- Note: Issues not discovered will be addressed during the customer query phase (during the rescore 
window). 

- Unreported issues or problems not flagged that are related to processing: 

Student tested in 2 books 

Students tested in each other's books 

Ripped, torn, or missing pages 

Wet, damaged, or contaminated materials not properly noted or packaged 
Manipulative, staples, loose pages, or scratch paper left in booklets 
Improper packaging of material: 

SGL not in Box 1 of each school 

Stack cover cards not numbered correctly 

Placing the GIS under the wrong stack cover card 

Boxes not numbered properly by school by corporation 

Scorable and non-scorable documents mixed together in the same box 

Using custom boxes to return shelf or 1LEPA material 
Materials sent for additional (unexpected) schools which were not in the database 
Late return of test materials received after the established cut-off dates 
Placing the wrong color return label on the box: 

Scorable materials packaged in blue label boxes 

Unused materials packaged with scorable materials 

- Not properly labeling "Do Not Score" material 
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Placing "Do Not Score" material with live scorable materials 
Incomplete shipments of returned test materials 
Missing expected school(s) 
Photocopied scannable documents: 

Test booklets 

Group Information Sheets 
SGL - GIS mismatches: SGL does not match the GIS 1 00%; SGL data does not match scanned data: 

Teacher name, grade, and student count 

Altered school name, school number, and corporation number 

Case count discrepancies - SGL and GIS do not match 
No SGL/no GIS received for processing 

Additional groups received but not listed on the SGL; teachers listed but not received; multiple groups 

listed but only 1 group received; one group listed and multiple groups received 

Students switching from online to paper/pencil and vice- versa, resulting in duplicate records 

In the event that these or other factors affect the reporting turnaround time, CTB will negotiate in good faith 
with the IDOE to deliver reports as soon as possible given conditions for each administration. 

Other Statewide Reports to IDQE 

Erasure Analysis Reports 

Research will analyze the Multiple-Choice erasures (both paper/pencil and online) and create an Erasure 
report for the IDOE. The report will be delivered approximately 2 months after report delivery for each 
administration. Specific dates will be determined in consultation with the IDOE. 

Impact Data Reports 

Research will analyze the validation student population data and generate an impact data report for the 
IDOE. This analysis encompasses a longitudinal analysis of statewide data using pre-equated parameters, 
based on the validation student population. A review of the impact data report and IDOE sign-off is on 
the critical path with the 5-day TAT and delays with a sign-off will result in report launch dates being 
pushed out. Scoring and reports processing will NOT begin until the signoff is received from IDOE. 

Technical Reports 

The Operational Technical Reports will be provided to the IDOE approximately 3 months after report 
delivery for each administration. Specific dates are yet to be determined. 

Error Report 

On a monthly basis, Program Management will collect data from all regions about the number and type of 
errors that have occurred on custom programs in the following categories. This information will be 
presented to the IDOE on a quarterly basis verbally without identifying the name of the custom programs 
involved. If the IDOE has any specific concerns that are not addressed by the explanation provided by 
CTB, CTB will request permission from the customer in question to provide contact information to the 
IDOE. The IDOE will sign a statement acknowledging the receipt of each verbal report. 

Test Content: A Test Content Error is defined as the inclusion of incorrect or inappropriate item(s) in an 
assessment, such that the validity or reliability of the assessment was affected, and correction of which 
caused significant delay of the program and/or resulted in public comment. 

Test Materials Production: A Test Materials Production Error is defined as an error in printing, such as 
incorrect pages or missing pages that affected the validity of the assessment or correction of which 
caused significant delay in production, such that the testing was jeopardized and/or resulted in public 
comment. 

Page 30 of 63 




Contract No. TBD 



Test Materials Delivery: A Test Materials Delivery Error is defined as an error in packaging, spiraling 
of booklets, or an error in the delivery of booklets that significantly affected the validity of the 
assessment or delayed the testing dates and/or resulted in public comment. 

Incorrect Information: Incorrect Information is defined as information that is printed in an assessment 
report that includes incorrect scores on student documents, incorrect school, corporation and/or state data 
and/or resulted in public comment. The cause of these incorrect scores must be found to be a result of 
errors internal to CTB and not as a result of school, corporation or state actions or failure to act. 
- Delay in Report Delivery: Delay in Report Delivery is defined as unapproved late delivery of reports 
that significantly affect a school, corporation, or the state's ability to meet internal and external deadlines 
and/or resulted in public comment. 



Document Retention 



- Used test materials will be retained as noted in the table below until approval is given by the IDOE. 

- Unused test materials will be retained as noted in the table below until approval is given by the IDOE. 

- DO NOT SCORE material should be separated from inventory material and retained with Operational and 
used data until approval is given by the IDOE. 

- Used materials will be retrieved upon request. 

Written authorization from the IDOE is required prior to destruction. 

Approval from the IDOE is required for all document destruction based on the following schedule: 



Grade 


Material 


Retention Time 


3,4, 5, 6, 7,8 


Operational - used + 
Do Not Score documents 


One year 


3, 4, 5, 6, 7, 8 


Non-consumable and unused 
materials 


6 months 


3,4, 5,6, 7,8 


Unfulfilled materials (left at 
vendor) 


30 clays after test window 
close 



CTB/Indiana Help Desk 



- Times available: M-F 7:30 a.m. to 5:00 p.m. EST 
Accepts and processes calls for short/adds 

Accepts and processes calls for Large Print and Braille materials and updates spreadsheet 
Accepts and processes calls for scoring issues, reports issues, and material shipments 
Accepts and processes calls for all activities relating to online testing - Registration and Technology 
Survey, Site Readiness and setup, Coordinator Trainings, Practice testing, Operational testing and post- 
operational testing 

Makes customer calls for information needed 

Provides updates for the weekly ISTEP+ team status report 

Creates WebEx training recordings 



Attachments 



For additional information please reference the attachments listed: 

Attachment A - Program Milestones and Critical Dates 

- Attachment B - OAS Work Plan 

- Attachment C - INORS Work Plan 

- Attachment D - The Parent Network Work Plan 

- Attachment E - Operational and Pilot Test Design and Development Specifications 

- Attachment F - CCSS Transition Online Pilot Test Design and Development Specifications 
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Attachment A 

To Exhibit E 
Program Milestones 

The following, with the exception of the test windows, are tentative critical dates for Work Scope hereunder. 
Key Deliverables that are subject to Liquidated Damages are indicated with a dollar sign ($). The tentative 
dates below are based on the original scope of work for the contract extension which does not include any 
adjustments that may be necessary for sell-up options chosen by the IDOE. The parties agree to negotiate all 
tentative dates during the initial planning meetings for that contract year and to update or revise the dates in 
accordance with Paragraph 8 "Changes in Work" of the Contract. 



Program Deliverables 


2013 


2014 


AnnliVH ^Ullc Tpct K/Tatprinfa Tn-Qitf» $ 

rVUUllCU Oivlllo 1 Col 1Y1CIICI lolo 111-oHv u> 


2/15/13 


2/14/14 


/VppilcU oKIIlf> SCSI WlUUUtV 


3/4/13 - 3/13/13 


3/3/14-3/12/14 


Multiple-Choice Test Materials In-site $ 


4/13/13 


4/11/14 


Multiple-Choice Test Window 


4/29/13-5/8/13 


4/28/14-5/7/14 


Multiple-Choice Online Test Window $ 


4/29/13 - 5/10/13 


4/28/14 - 5/9/14 


INORS Live (Phase 1)$ 


5/24/13 


5/23/14 


Image Print Live (Phase 1) $ 


5/24/13 


5/23/14 


Parent Network Live (Phase 1) $ 


5/24/13 


5/23/14 


Student Reports In-Site (Phase 2) 


6/7/13 


6/6/14 


Student Report Labels in-site (Phase 2) 


6/7/13 


6/6/14 


INORS Summaries Live (Phase 3) 


6/7/13 


6/6/14 


Post-Rescore Reports Live (Phase 4) 


7/19/13 


7/18/14 


Post-Cleanup Reports Live (Phase 5) 


9/16/13 


9/15/14 



Page 32 of 63 



Contract No. TBD 



Attachment B 



i 



To Exhibit E 
ISTEP+ Online Assessments Work Plan 



CTB will provide ISTEP+ assessment in both paper and online modes for English/Language Aits and 
Mathematics in Grades 3-8, Science in Grades 4 and 6, and Social Studies in Grades 5 and 7. 

Online Test Administration Parameters 

Changes from the previous year / contract: 

• As of September 2011, the customer-mandated online testing for all schools for at least the two 
highest grade levels provided the technology and bandwidth supports. 

• The mandate will drive an increase in the online testing population to at least 350,000 students across 
the 6 grade levels. 

• It is anticipated that portions of the Applied Skills assessment will be administered online in Spring 
2013 and Spring 2014. A final decision will be made based on the outcome and information that is 
available from the Spring 2012 Applied Skills Online pilot. 



1 . The CTB online testing capacity set aside for Indiana is for up to 70,000 simultaneous non- 
accommodated students. Up to 5% of the students testing online in Indiana are expected to use the Text 
Reader accommodation. 

2. Participation is contingent upon schools/corporations meeting minimum hardware and software and 
network requirements. 

3 . Site Readiness reviews can only be conducted during school operating days and hours. 

4. Scheduled holidays, institute days, staff development days, and school- and/or corporation-mandated 
tasks may preclude staff availability for Site Readiness reviews as well as training for OAS. 

Assumptions 

1 . OAS upgrades requested by DOE for features not currently available in the system will he handled as 
change requests. 

2. It is assumed that only a single mathematics reference card is required for Grades 6-8. 

3. Site Readiness reviews are assumed to take 4 hours each, with 2 visits accomplished in a single day. 

4. Most Site Readiness preparation will be done remotely, with onsite support for up to 45 school sites per 
year. CTB staff will visit the 45 school sites for onsite support. 

5. School sites will need to download the student test client as a part of the Site Readiness work. 

6. It is assumed that the OAS must provide the capacity to test approximately 80% of the total population, 
i.e., 400,000 students testing using the online mode. 

7. It is assumed that Large Print and IEP-CD accommodations testing delivered in a paper/pencil mode are 
matched to the extent appropriate by equivalent accommodations currently available in OAS, such as 
large font and text reader. 

8. It is assumed that slight look and feel variations are expected when the paper/pencil content is presented 
online. 

Operational Online Administration Spring 2013 

For the Spring 2013 administration, CTB will provide Online Assessment services to test about 5% of the 
student population for the Applied Skills assessment and about 70% of the student population in grades 



Constraints 
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3-8 for the Multiple-Choice assessment. Key services and related deliverables are described in the 
following sections. 

CTB will follow the OAS Implementation methodology that provides for constant communication with 
the IDOE on the planning and definition of tasks and activities. It includes extensive internal and client 
reviews of all custom deliverables, leading to the approval of veiy high standards and full customer 
satisfaction. As part of this methodology, the following key services and deliverables are included in 
scope and described in the following sections: 

• OAS Implementation Project Management 

• Content Production Support 

• Client Specific OAS Configuration 

• Data Provisioning: Student Pre-Identification Data and OAS User Data 

• Training and Documentation 

• Implementation Services: Site Survey, Site Readiness and Certification, and Test Administration 
Support 

OAS Implementation Project Management 

Standard project management processes are employed for planning, executing, monitoring and control 
and closing the project. A detailed implementation plan and schedule will be developed for each year of 
the contract. The project execution will be closely monitored to ensure the project objectives are met on 
time and within budget. An integrated change management process will be used to ensure that negative 
impacts of changes during the life of the project are reduced. The project risks will be analyzed 
periodically and mitigation plans will be identified and risks are managed to closure. 

Client Specific OAS Configuration 

OAS is a configurable system that allows setting certain features, as needed by the customer, without any 

software code changes. 

These configurations may include: 

• organization hierarchy definition and label names 

• user roles and assignments 

• student demographic labels and values; student pre-identification data protection 

• accommodations settings for background and font color changes controlled by the test 
administrator, calculator, text reader, un-timed test 

• product acknowledgment page 

The configurable elements in OAS will be reviewed in detail with the IDOE and, as needed, screen mock- 
ups will be provided to confirm the settings. A business and configuration requirements will be created 
and will be used to apply the configuration settings to OAS prior to the test administration. 

Content Production 

For Content Production, OAS Implementation and Development will provide support to develop the 
content framework, promote the content to the various review environments and provide the content 
review web-based tools. This support will be provided to produce, for each administration year, one 
content form per grade per subject, as well as two - three practice test forms per administration year. 
Refer to the Publishing scope for details regarding the content scope and forms design. 

Data Provisioning 

Data provisioning is done to preload the students' data into the system. 
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For client acceptance purposes, a simulated corporation and school will be identified and created in the 
acceptance and production environment. The data for user acceptance will be exported for scoring and 
reporting and provided, along with the paper/pencil live data, after the test administration. 

Student Pre-Identification Data 

OAS will make use of the student barcode files to select and load students for the grades indicated in 
Online Test Administrators Parameters. The 1DOE will review the student data in OAS prior to 
opening the system for general school use. The student preloaded data can be modified in OAS, and 
the users can also create new student data. Screen shots will be provided to confirm the demographic 
labels, positions and other graphic user interface aspects (e.g., using checkboxes, dropdown lists, and 
radio buttons for values selection). 

As in previous years, the pre-id data cannot be modified in OAS, and the student biographic/invalid 
STN updates will prevail. Due to the timing of the updates, they would not be applied until just prior 
to the data extract from OAS. 

For all the confirmed sites and grade combinations, student tests will be pre-assigned (pre-scheduled) 
based on the pre-id data loaded. 

OAS User Data 

Pre-existing CTB SSO user accounts are enabled in OAS for eveiy staff member identified during the 
school selection process and according to their specific roles. New accounts will be created as needed, 
as well. These OAS accounts are used by the technical staff, test administrator staff and test proctors 
to access the system and perform the activities permitted for each user role. 

Training 

Training materials will be provided to the Indiana staff to ensure a successful implementation. All of the 
training materials will be provided in electronic form and posted online at <www.ctb.com/istep> . 
Aside from training materials described in the table below, a pre-recorded practice test for students will 
be made available. 



The following training sessions will be provided: 



Component 
Training 
Participants 


Test Administration Overview 


-Technical Coordinator Systems 
Overview 


Subject of the 
training or 
professional 
development 


Introduction to the Summative Assessment 
Program; Overview of OAS features and 
functions; Training in Administrator tasks 
and responsibilities 


Introduction to the OAS system; 
Overview of technology architecture 
and implementation procedures; Set- 
up of Administrator and Student 
Work Stations 


Training format 


Hands-On; Online conducted on-site with Webinar available as a make-up; 
Demonstration and oral presentation Train-the-Trainer 


Total number of? 
sessions 


8 half-day onsite sessions within a 2-week period (2.5 hours each) 
1 Webinar sessions within a 2-3 week period (2 hours each) 


Total number of 
educators pur 
session 


Onsite sessions: ~20 per session 
Webinars: ~200 per session 


Total number of 
trainers per session 


Up to 2 
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Component 
Documentation 
provided 


7 : : ■ y -y : y--y&' : y^y^ - 

PowerPoint presentations, OAS User's Guide, ISTEP+ Online TEM, Handouts, 
Technology Checklist 

(A Digital OAS Admin Toolkit can be downloaded from the 
<www.ctb.com/istep> site.) 


Additional 
Training/Material 


All material posted online 

Make-up web-ex, performed and posted online 


Training Locations 


8 regional Training sites throughout the state will be selected with the help of the 
IDOE (depending on participation) 



Documentation 

OAS includes standard online help in the system, as well as a standard user guide and an installation 
guide. These guides are distributed in PDF format. 

The custom Indiana OAS Quick Start Guide, which was produced in 2009, will be updated as required 
and re-published in 2013 and 2014. The Online Examiner's Manual (OEM) will include test 
administration instructions for Test Examiners. 

Implementation Services 

As part of CTB's implementation services, CTB's technology surveys will be distributed using online 
applications in order to identify the technology at each school considering participation in ISTEP+ 
Online. The sites will be grouped by their ability to meet requirements {Fully Meets, Conditionally 
Meets, and Does Not Meet). Support to help sites achieve readiness would include documentation, 
emails, teleconferences and site visits, as appropriate. A final readiness status will be provided to all sites, 
as well as to the IDOE. 

Site Readiness 

Based on our experience, paying close attention to these final steps prior to testing is critical in order to 
provide a reliable testing environment and prevent issues during the test administration. 

At the time the training is completed for the schools that are preparing to test, the supporting 
documentation for the system will have been provided to participating schools. The schools' technical 
staff will be involved to configure and confirm that the workstation set-ups are completed, and the 
corporation technical staff should work with school staff to ensure network configuration requirements 
are fulfilled. Checklists are provided to guide the school and corporation staff during these activities. At 
the schools visited by CTB's technical staff, these activities will be performed by the school and 
corporation staff respectively, as monitored and assisted by CTB's field engineers. 

Users who are setting up student workstations can log into OAS, where utilities will automatically detect 
whether correct versions of Java and Flash are on the workstation. School technical staff would 
download upgrades as necessary. 

At this time, the enciypted tests must be downloaded and pre-positioned on the students' workstations. 
Site Certification Practice Test sessions should be scheduled. Each site must confirm to CTB that all of 
the Site Readiness activities were successfully completed. Only when this final step has been completed 
can the online assessments begin for that site. 

Based on data gathered from previous administrations, there are about 1800 buildings within the state that 
service at least one grade in the 3-8 grade span. 
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Registration for Spring Online testing 

• CTB prepares the Site Technology Survey and Registration form 

• CTB distributes the electronic Site Survey to all interested sites. This survey is to be completed by 
each site's technology coordinator and returned to CTB within 10 business days of receipt. 

• The results will be used to adequately estimate which schools meet the OAS Configuration 
Requirements and are pre-qualified to participate in the online testing program. 

• CTB follows-up with sites. Should any sites indicating interest to test online fail to send 
complete/accurate information, CTB will schedule phone conferences with the site and/or corporation 
technology coordinators in order to clarity and rectify any outstanding issues with the survey. 

• Should CTB determine that the site is not (or may not) be ready for OAS testing based on RTS data, 
the site will be advised that their equipment needs to be upgraded and they may plan to test only after 
the upgrade is completed, or an onsite visit may be scheduled to prepare the site for testing. 

• The OAS Implementation Services Manager will be responsible for delivering the following: 

■ School Technical Capabilities Report 

■ Recommendations for schools not fully meeting online testing requirements 

■ List of Registered Schools for online testing 

■ List of Contacts that require accounts for online testing; we will add one tech 
coordinator and one test coordinator per school and one tech coordinator and one test 
coordinator per corporation. Corporation staff will be trained on how to add users. 

Test Administration Product Support 

Product Support for OAS users will be provided during the hours of 7:30 am -5:00 pin EST. OAS 
Product Support will provide real-time support to each school that is administering the online test. 
Product Support will answer questions and work with users in resolving issues regarding the testing 
process, system procedures, error conditions, student testing status and system status. Ongoing 
administration reports will be provided to the IDOE. 

OAS Product Support is backed by CTB's network, development and technology support teams so that all 
customer calls can be handled expeditiously. AH customer calls are logged and tracked to enable analysis 
of call handling and problem resolution statistics, resulting in increased process and system improvement. 

Scoring and Reporting 

The student test data will be provided to other systems at CTB for Scoring and Reporting, along with the 
paper/pencil tests. The details for these activities are described in the Scoring Scope and Reporting Scope 
documents and are not part of the present document. 

Out of Scope 

1. OAS Manipulatives: any manipulatives not currently available (such as coins or shapes) are not in scope. 

2. Training: For training-related travel, CTB will cover the travel and accommodation costs for CTB staff. 
CTB will not be responsible for the cost of the educators' travel to regional training sites. 

3. Pass-fail raw scores visible to students at the end of the test. 

Security 

OAS employs industry-standard best practices to ensure security of test content, the testing experience, 
and the student reporting data. The system operates in Web-based mode, with all communication 
between the OAS server and the student workstation over HTTPS. 

The test content is securely stored in an encrypted object bank using 256-bit encryption. Encryption keys 
are not retrieved from the server until the student successfully logs into a test, and the keys are only 
placed in temporary memory. The encrypted test items stored on the hard drive are only decrypted at the 
moment a student requests the test item. The item remains encrypted at all other times. 
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The testing experience is provided in a browser window that is locked-down on the student desktop. The 
browser window uses the full screen, overlaying any task bars or other control elements on the computer 
desktop, and cannot be minimized. This prevents students from accessing anything other than the test 
delivered by the browser. Students only have access to the testing experience with a proper login ID and 
password, and these are unique for every test session to which the student is assigned. Cheating is also 
discouraged by including features such as random test form assignment. Further, should a student be 
suspected of a testing impropriety, the administrative interface allows a student's test to be invalidated 
(and re-validated) at the click of a button. 

Access to the student data is strictly controlled through a system of user roles and assignments in the 
organizational hierarchy. Users may only have access to the data and functionality appropriate to their 
role, and only for the members of the organization to which they are assigned. 

Test administrators overseeing the students during testing can monitor student test status in real time 
using the OAS administrative software, and (depending on each organization's rules and procedures) can 
invalidate test results for students suspected of cheating without losing the student's results or disrupting 
the testing process. Security is maintained in the OAS administrative software by means of hierarchy 
controls at various access levels, using a structure of limited permissions and login passwords. 

To ensure business continuity, OAS uses redundant fail-over resources at the database, application server 
and web server layers. OAS is hosted by Equinix EBX centers, which set the industry standard for secure 
and reliable collocation services. Equinix has a histoiy of delivering robust and reliable data centers in 
the industry. OAS is hosted by McGraw-Hill's Global Operations and Infrastructure Division, the group 
that also hosts Business Week, Standard and Poors, and Platts. The data center has extensive backups and 
recovery procedures, and was operational through both the 9/1 1 tragedy and the New York City blackout. 
OAS is prepared also for situations when the entire Data Center is affected by natural disasters, such as 
earthquakes or hurricanes. A disaster recoveiy plan and resources are in place to ensure the business 
continuity in case of a catastrophic event. 



Project Deliverables 



Deliverable 


Description ~ 


Produced By 


Stakeholder Signatures 
. Required 


Project 
Schedule 


A set of detailed milestones that 
identify when the client 
resources are required for 
reviews or other activities 


Program 
Schedule 
Analyst 


IDOE and Program 
Management 


Content 
Requirements 


Documentation on framework 
code, commodity code, 
assessment title, deliverable unit 
titles, forms, level/grade, time 
limit, product name, number of 
subtests 


Publishing 


IDOE and Program 
Management 


Practice Test 
Content 


Set of custom test items to be 
used for student practice testing 


Publishing 


IDOE and Program 
Management 


Operational 
Test Content 


Set of test items to be used for 
operational testing 


Publishing 


IDOE and Program 
Management 


Technology 

Schools 

Selection 


A list of schools that have the 
technology and infrastructure to 
test online 


Impl Mgr 


IDOE and Program 
Management 
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Del ivei able 


Description 


Produced 




Weekly Call 

Tracking 

Report 


Excel workbook with detailed 
statistics on customer calls and 
corresponding quality metrics, 
including trend 
analysis/summary 


Cust Support 


IDOE and Program 
Management 


Technical 
Training Slides 


Presentation slides to be used for 
Technical Training 


Training 
Mgr/ 

Subcontractor 


IDOE and Program 
Management 


Administrator 
Training Slides 


Presentation slides to be used for 
Administrator Training 


Training 
Mgr/ 

Subcontractor 


IDOE and Program 
Management 


Standard OAS 

Installation 

Guide 


Standard Installation Guide and 
Checklists 


OAS Tech 
Writer 


NA 


Standard OAS 
User Guides 


Standard reference document for 
users who will be responsible for 
managing users, adding or 

mnrllfvinci cluHpnt Hata 

scheduling tests, and proctoring 
tests 


OAS Tech 
Writer 


NA 


OAS Quick 
otan ouioe 


Short guide on main functions 

/1**cr*t*f iff tnf* 01/ off* m 


OAS Tech 
Writer 

TV 1 H\st 


IDOE and Program 
Management 


TEM — online 
section 


Test Examiner's Manual 
provided in PDF format 


Publishing 


IDOE and Program 
Management 


OAS Business 
and 

Configuration 
Requirements 


ISTEP+ Business and 
Configuration requirements 


CTB 

Requirements 
Analyst 


IDOE and Program 
Management 


Student Data 
Load (PrelD 
Load) 


Student data, including 
demographics; data typically 
from pre-code 


ETL Dev 


Product Support, 
optional IDOE 


User Accounts 


User ids and passwords to allow 
state and school officials OAS 
system access, limited by the 
user's role (administrator, 
coordinator, accommodations 
coordinator, proctor) 


Cust Support 


NA 



Critical Success Factors 



Critical Success Factors 



Success Definition 
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Critical Success factors 


Success Definition 


Project Schedule 


Delivery of the following components a minimum of 2 weeks prior to the 
test: 

o OAS platform configured for Indiana 

o OAS loaded student barcode data 

o ISTEP+ Online Practice and Operational Content forms 

o Test Administrator's Manual 

o Technical and User Training 

Site registration for online testing by end of November 

Site Certification activities completed at least 3 days prior to the practice 

testing 

Approvals for any deliverables are received within a maximum of 5 days 
from the IDOE. 

Online testing is completed in time to facilitate the 7-day TAT for reports. 


Project Scope 


Delivery of the following critical components: 

o Schools selection based on technology criteria and online 

testing registration response 
o OAS configured for Indiana 
o ISTEP+ Online and Practice Content forms 
o Test Administration Manual 

o Technical and User Training materials, including the 
Quick Start Guide 


Project Quality 


The delivery of this project within CTB's metrics is a critical success 
factor for this project. These metrics include: 

o Schedule Variance - Non-critical Components within a 

+/- 20% variance from the base-lined project schedule, 
o Schedule Variance - Critical Components ~ Zero 

variance for components on the operational test 

administration critical path, 
o Effort Variance - Delivery of the project within a +/- 5% 

variance from the base-lined definitive estimates, 
o Post-Production Defects - No priority 1 or 2 defects 

post-production. 


Resources 


IDOE representative is available to facilitate response and resolution to 
Mate-related issues/ decisions wiiniii ine agreeu nine irdiiie. 

IDOE and school staff will be available and responsive to CTB's 
inquiries for site survey, certification and readiness. 

CTB is provided access to school and corporation facilities in order to 
perform site certification-related activities. 



Project Acceptance 

Prior to the release of the system to the participating schools, CTB will provide the IDOE with access to 
the system in a User Aeceptance Testing environment for acceptance and sign-off for the Content, OAS 
Platform and configurations for Indiana, as well as a review and sign-off of the loaded schools hierarchy 
and student data. 

Every effort is made to ensure that at this stage no critical or significant issues are present. It is expected 
that any non-critical issues encountered at this time will be addressed by including further information in 
the training materials, while a long-term solution will be provided in the following administration. 
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The User Acceptance Testing should be conducted in the final production environment in a UAT- 
dedicated corporation and schools. The users will take tests as students, and the results will be provided at 
the end of the testing, along with all other student results. 

Content Acceptance 

The final Practice and Operational content user acceptance review is done using the OAS Test client in a 
User Acceptance Testing environment that replicates the OAS production environment. At the time of 
this final review, the content has already undergone several reviews and all critical issues were detected 
and addressed. At this final review, the content should be used during test taking as students and only 
major issues, if any, may be addressed if possible in the short time frame remaining until the test 
administration. Any minor issues identified at this time will be addressed during the following 
administration. 

OAS Platform and Configuration Acceptance 

The OAS platform and the configurations specific to Indiana will be provided for a final client review and 
the following criteria will be used for Acceptance. 

The OAS platform and configuration are deemed accepted when the Test Administration System and the 
Test Client configurations are set according to the ISTEP+ business and configuration requirements for 
data hierarchy configuration, user set-up, student demographics and accommodations. 

Student Data Acceptance Criteria 

The IDOE will verify that the number of students expected to load have been loaded and will do a spot 
check of the student demographics data for the students included is loaded in OAS. 

Localized student data issues or missing student data affecting less than five students in a school location 
will be addressed by the IDOE either correcting or creating new student data using the OAS system. 
A RES update will be applied before the online testing students' data is merged with paper/pencil testers 
for scoring and reporting. 

Technology Requirements 

The technical requirements listed below are based on the OAS 9 Installation Guide. In order to 
participate in the ISTEP+ Online, schools must meet these requirements. 

These requirements may change until the time of the test administration every year, as it is expected 
that there will be new application upgrades. The upgrades are carefully planned to keep the system 
updated with recent technology advances and to facilitate more extensive participation in online testing. 
As more information becomes available, CTB will inform the IDOE, and as appropriate, corporations 
and schools, about changes in requirements. 

REQUIRED COMPUTER EQUIPMENT 

Each computer must meet these minimum specifications: 

• Platforms: 

Windows PC with 1 .3 GHz CPU and Windows XP Service Pack 3 or greater, Windows 
2000 Service Pack 4 or greater, or Vista Service Pack 1 or greater OR 
Intel Macintosh with 1 .8 GHz CPU and 1 0.4.8 OS or OS 1 0.5 .4 
■ Linux (Suse, Ubuntu, Rehat) 

• Hard disk space: 200 MB available disk space for object bank 

• Memory: 512 MB RAM 
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• Internet access: lOmb Ethernet connection or equivalent wireless connection 

• Bandwidth: 1.5 Mbps upload and download - Tl line, high-speed cable or high-speed DSL for 
eveiy lab with 50-100 concurrent users 

• Software: 

■ Java Runtime Environment (JRE): Java 5 .0, Update 1 4 or later for PC or 

■ Java Runtime Environment (JRE): Java 6.0, Update 10 for Vista on PC to download test 
content ahead of time 

■ Java Runtime Environment (JRE): Java 5.0, Update 7 or later for Mac 

• Test Administrator's Workstation: 

IE 6 OR IE 7 (Windows) 

■ Firefox 2.0. Note: IE must be used to download test content or install Flash on student 
workstations. 

Safari 2.0 (Mac) 



• Student Workstation: 

■ IE 6 OR IE 7 (Windows) Must be resident on workstation 

Safari 2.0 (Mac) 



Each school must provide a sufficient number of computers that meet the minimum requirements in 
order to conduct practice sessions prior to testing. 
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Attachment C 
To Exhibit E 

Indiana Department of Education Online Reporting System (INORS) Work Plan 

Overview 

In the Spring of 2013 and 2014, ISTEP+ reporting will be delivered through the INORS website solution. 
The INORS solution will be released in iterative stages (phases) throughout both the 2013 and 2014 
administration report releases and will include reporting for the ISTEP+ Assessment in Grades 3-8. 

The following table provides a high-level description of the features included in INORS: 

I iidiana Online Reporting System 

• Online Reporting for ISTEP+ Assessment 

• PDF Report Downloading of Student Report PDFs (current plus one previous administration) 

• PDF Image Print of Individual Student Responses from Paper/Pencil Testing (current plus one 
previous administration) 

• Group Download PDFs - Student Report PDFs and Image Prints of Individual Student Responses 
(current plus one previous administration) 

• Invitation Code PDFs - Parent Network Invitation Code Letter PDFs (current administration) 

• Invitation Code PDFs Batch Downloads - Parent Network Invitation Code Letter PDFs (current 
administration) 

• Online Dynamic PDF Generation of All Online Reports (from Spring 2010 forward) 

• Online Rescore Request Form (current administration) 

• Corporation- and Nonpublic School-level .csv GRT File Download (from Spring 2010 forward) 

• Corporation- and Nonpublic School-level .csv Invitation Code File Download (current 
administration) ____^__________ _________ 

• Corporation- and Nonpublic School-level GRT File Record Layout Download (from Spring 2010 
forward) ____ _____ 

• Corporation- and Nonpublic School-level Invitation Code File Record Layout Download (current 
administration) 

• Secure role-based access, using the Single Sign-on user accounts for State-, Corporation- and 
School-level users 

• Administration specific teacher-level accounts to access classroom-level data, based on teacher 
names used on the Group Information Sheets (GIS) 



Included below is a more detailed specification of the solution and implementation timelines. Spring 2013 
and 2014 reporting will be consistent with deliverables and processes established for the Spring 2012 
administration. 
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Included Datasets 

The datasets that will be included in the production solution for the Spring 2013 and Spring 2014 are: 

• ISTEP+ Assessment Grades 3-8 

• Student Report PDFs 

• Image PDFs of Individual Students Responses from the Applied Skills test 

• Parent Network Invitation Code PDFs 

Additionally, as a part of the Spring 2013 and Spring 2014 INORS solution, the Spring 2010, Spring 201 1 
and Spring 2012 ISTEP+ Assessment results data previously loaded into the INORS will be available (see 
assumptions section below). ISRs and Image Print PDFs will only be available for current and one previous 
administration. 

Implementation Timeline and Overview 

The implementations of INORS will consist of four (4) phases: 

• Planning 

• Development 

• Stabilize 

• Deploy 

Each phase has a defined purpose and scope and will act as a framework to ensure that IDOE expectations are 
properly met as an updated version of this solution is introduced to corporations and schools. The dates listed 
below are for estimation purposes only. Upon contract execution, a detailed INORS project plan will be 
finalized and used to communicate dates, timelines, and milestones during this project. 

Defects that may be identified in INORS will be addressed as they arise. Requested changes to the 
functionality, or functionality/reports that are not agreed to during the planning phase for each year will be 
handled via a change order. CTB will work with the IDOE to ensure that all requests are prioritized and 
handled as effectively as possible, while limiting time and resource requirements from all parties. It is 
important to have requests communicated in advance so any changes can be properly scoped, developed and 
tested early enough to avoid impacting the reporting window or INORS system functionality. 

Goals 

For each administration, the goals for the reporting solution are: 

• Deliver a fully functioning Online Reporting System (ORS) to the Red-Team and Production 
environments for review and approval by the IDOE. 

• Provide online reporting and online report delivery in a timely and accurate manner to all users. 

Functionality 

This section describes the basic functionality that will be included within INORS. A more detailed discussion 
of the exact reports is included in a later section. INORS is a browser-based system that is designed to deliver 
online reporting to the state of Indiana. The purpose of INORS is to enable educational leaders to make 
informed decisions about curriculum, school programs and facility needs, based on real data. INORS 
provides assessment data for the ISTEP+ test at the State, Corp/Diocese, School, and Grade level for Indiana 
public and nonpublic schools. 
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Frequently Asked Questions fFAQ) 

A FAQ section is provided that can be used to answer basic questions that may need to be addressed. 
Help 

An online help section will provide users with searchable help that can be used to answer most usability and 
technology questions. The help system will additionally include tutorials and answers to basic questions. 
Help Desk support information is included in the Support section below. 

Manage Users 

The "Manage Users" section will allow administrative users the ability to manage ail user accounts within 
their authoritative domain. Administrative users have the ability to manage all descendant accounts within 
the hierarchy (i.e., a corporation administrator can manage corporation user accounts and both school 
administrative and user accounts). The only exception to this rule is the state administrator role. State 
administrators can manage all accounts (even other state administrative accounts) within the system. Manage 
users also includes the ability to create username and password sheets that can be distributed to INORS users. 

User Settings 

The user settings functionality is for individual users to manage their own account. From this interface, users 
can change their password, update their email address and set the answer to their seeurity/reminder question. 

Reports Area 

This section will include the available selection of reports that a user will have access to within INORS. A 
more detailed description of the reports is included below. 

Downloads Area 

This section will include all data files that are accessible for download by a particular user. 
Corporation level GRT and Invitation Code file downloads will be available within INORS to state- and 
corporation- level administrators. Nonpublic school-level GRT and Invitation Code downloads will be 
available within INORS to state-, corporation-, and nonpublic school-level administrators. 

For each test administration, deployment will also contain the Individual Student Reports (ISR) PDFs, the 
Image Print of individual student response PDFs and the Parent Network Invitation Code PDF letters. 

The Group Download option will continue to allow users to select ISR and/or image print PDFs for multiple 
students to be combined into one file, downloaded and printed. An Invitation Code PDF letter batch download 
option is also available. 

Tools Area 

The tools area provided within the INORS website can provide users with access and links to additional 
materials, such as training and interpretive guides. During the finalization of scope and design, the INORS 
team will work with the IDOE to identify any electronic materials that should be made available through the 
INORS website. 

Informational Area 

At the bottom of the INORS website, an informational area exists that will display appropriate copyright, 
contractual and informational items. Legally, CTB must include copyright details, terms of use policy, online 
privacy policy, and applicable Children's Online Privacy Protection Act (COPPA) information. Additionally, 
customer support phone numbers and email address will be included. 
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Pleader/Messaging Area 

At the top of the INORS website, a header and messaging area will be utilized to display messaging 
information. This area will include a small CTB/McGraw-Hill text and "The McGraw-Hill Companies" 
image in the upper left and right coiners respectively. Additionally, this area will include a "logged in as" 
notice which will indicate the name of user logged into INORS. This area will include the appropriate IDOE 
logo, along with any messaging and naming that is required. 

Report Details 

Along the bottom of the menu, on the left side of INORS will be a Report Details section. This section is 
used to include informational material about the reports, such as scale score ranges, privacy notices, report 
legends, report purpose text, etc. The Report Details section is configurable for each report and will be 
defined during the planning stages. 

Backer Text/More Information 

A "more info" button or hyperlink will be included with each report and will contain the traditional backer 
text that would normally appear on the back of a printed report. Backer text is customizable per report and 
will be updated in consultation with the IDOE. 

Print Report 

Each online report, as it is viewed on the screen, will be capable of rendering in a PDF format through the 
"Print Report" button or hyperlink. 

Report Header/User Control 

Each online report will include a user control that will enable users to navigate through the available options 
for the report based on the report and their level of access to the system. Options within the user control will 
vary by report, but the control will use "sticky" filters and maintain prior selections while navigating through 
the reports, where practical. 

Embargo Notice 

It may be necessary to include an embargo notice on reports based on data that has been officially released by 
the IDOE. It is our assumption that CTB will need to include this embargo notice and that we will need to 
easily and quickly be able to remove it from the reports, once the information has been released. The details 
of the embargo notice will be further finalized during the planning stages and incorporated into the design of 
INORS. 

Usage Reporting 

Basic usage reporting will allow administrators the ability to see when, if ever, a user last logged into INORS 
via the Manage Users functionality set. 

Homepage 

INORS will continue to include the existing homepage. Any changes to the existing homepage will be 
finalized during the planning stage. Design or layout changes to the homepage will be considered out of 
scope. 

Acceptable Use Policy 

The existing acceptable use policy designed by the IDOE will continue to be included within the system. 
This Online Usage Agreement may be reset periodically by state administrators and is typically used to 
educate ORS users on their responsibility for securely managing data and reporting. 

PDF/Print-On-Demand Reports 

Student Report PDFs will be available for download within INORS. These PDF reports are made available to 
users based on their login credentials. See the Reports section below for a detailed list of PDF reports that 
will be included. 
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Secure, Scoped User Access 

All users will access the system by providing a valid username and password. All user accounts are directly 
linked to a level within the organizational hierarchy that will not allow users to navigate to information that is 
external to their access level. For example, all school level user accounts are linked directly with an 
individual school contained within INORS. This allows school users the ability to see all pertinent 
information about their school and only their school. State and System comparative numbers may be 
included for informational purposes within the report, but the users will never have the ability to gain access 
to any other identifiable information about students that are not within then- school. 

Reports 

Reference Table 10 - Reports and Electronic Files Matrix 
Reporting Phases 

Reports are released in multiple phases as outlined below. 

• Phase - Invitation Code letter PDFs (approximately a week prior to the Phase 1 reports launch) 

• Phase 1 - Individual Student data, GRT Download, Proficiency Roster, Undetermined Status Report, 
Online Rescore Request form 

• Phase 2 - Printed ISRs and Student Labels in-site to corporations/nonpublic schools 

• Phase 3 - Summary reports and complete the reports release for the administration 

• Phase 4 - Updated reports from post-rescore window 

Support 

All customer and technical support will be handled by CTB/McGraw-Hill's customer support organization. 
Support requests can be made via a toll free 1-800 number or through email. INORS will include support 
contact information in the system, on the homepage and on all distributed material. 

Security 

CTB's ORS solutions use industry standard best practices to ensure data and system security. The ORS uses 
128-bit Secure Sockets Layer (SSL) over HTTPS to ensure that no private data is intercepted during 
transmission. 128-bit SSL is used by online banks and credit card companies, so it is both secure and reliable. 
The INORS solution is hosted at McGraw Hill's East Windsor facility. 

Availability 

The INORS website will be available via an internet browser 24 hours a day, seven days a week. At times, it 
will be necessary to perform routine system maintenance, and CTB will notify the IDOE prior to commencing 
with any regularly scheduled maintenance. CTB will attempt to schedule all maintenance during off-peak 
hours (evenings and weekends) so as to provide as little disruption to regular service as possible. Should an 
unexpected system outage occur, the INORS team will use all practical resources to return INORS to a usable 
state as soon as reasonably possible. 

Assumptions 

• Online reports will include class-level reporting. Interactive online reports will be available by grade 
at the class level and higher, as applicable. 

• Class breakdowns will be based upon the teacher of record within the data (GIS/GRT data). 

• Not all of the reports will be made available in an online format. Certain reports will continue to be 
delivered on paper (Student Label and Perfect Score report). See the Reports section above for a list 
of reports that will be available online. 

• PDF/Print-On-Demand Reports will continue to include the Student Report and Image Prints. 

• Longitudinal reporting costs are not planned, expected or included in this Work Plan. 
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• All user accounts are linked to Single Sign-on functionality. Current INORS functionality allows the 
IDOE to force a password reset when an employee of the state is no longer employed or at any time a 
State Administrator chooses to do so. 

• Interactive report data, including GRTs, will be retained for more than 2 years and only TSR-PDFs 
and Image PDFs will be retained for the current and one previous year. 

Constraints 

• Ail existing datasets will drive the online reporting. No new summarizations or aggregations (either 
pre-done or dynamic) will be available for INORS. 

• State-, corporation-, school-, and teacher-level users will have access to INORS. 

• Class and student level reporting will be available by the teacher of record, based upon GIS data (only 
the teacher on the pre-coded labels or bubbled on a test booklet). 
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Attachment D 
To Exhibit E 

The Parent Network and Individual Student Report Work Plan 

Individual Student Reports will be provided to parents and to educators electronically as PDFs. The Indiana 
Parent Network provides parents with student ISTEP+ results and guidance for working with students. 

Deliverables 

• Provide Indiana parents with Student Reports and resources online 

• Provide Indiana parents with a PDF copy of their child's Applied Skills responses (linage Print PDF) 

The Indiana Parent Network 

1 . Online secure access website for families of ISTEP+ assessment results and download of Student 
Reports 

a. Grades and subjects: 

i. E/LA and Math: Grades 3-8 

ii. Science: Grades 4 and 6 

iii. Social Studies: Grades 5 and 7 

b. Launch after state release of results to corporations and schools 

c. Invitation Code letters (login letters) generated for parents (one for each student) and 
delivered to corporations and nonpublic/charter/choice schools. 

i. Parent Network also provides functionality for schools to generate Invitation Codes 
themselves. 

d. Graphic display of score results at overall and standard level 

e. PDF download of Individual Student Report (ISR) on the website for parents 

i. Updates made to student records if scores have changed after the rescore period 

f. PDF download of student's Applied Skills test responses (Image Print) on the website for 
parents 

g. Links to Indiana instructional resources and released items 

h. PN will re-launch after each administration to include data from the current test 
administration, as well as previous test administrations. 

i. Customer Support provide via phone and email 

i. Hours of operations will be 8:00 am - 6:00 pm EST. 

ii. Response Time for Severity One issues will be within 24 hours of issue arising. 

Student Open-Ended Response Images (Image Print) on Parent Network 

1 . Parent Network hosts scanned "Image Print" files with images of student work 

a. File will be in PDF format (assumption is -500K per assessment year) 

i. Each file represents a single student's result images (all subjects in one file) 

ii. Assume naming convention based on Applied Skills barcode, grade and subject 

iii. Assume files provided in file directory collated by grade, subject, and school 

b. Files transmitted using a secure FTP or secure shared server after reporting window 

c. Image Print PDFs provided on the Parent Network website as PDF download 
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d. Parent Network will need to re-launch to include Image Print PDFs in addition to ISR PDFs (one 
per student for both) 

e. Image Prints available on the Parent Network for all students tested in Grades 3-8 (including 
those students who test online). 

Spanish Translation 

1 . Translation Guides to the Individual Student Report (in English and Spanish) in PDF, not printed 

2. Translation of the Parent Letter sent to parents into Spanish 

Assumptions; 

• Item level data, full-item release, and record of student work "image-print" are utilized for the Parent 
Network release. 

• Files of associated barcodes with pre-codes assigned so that the barcodes are utilized for matching 
with the GRT. 

Schedule Assumptions 

• Internal CTB Red Team Review of Parent Network website 

• A final review and sign-off of Parent Network by the IDOE will occur prior to release/before 
operations begin. 

o CTB will work with the IDOE to define the final Red Team Review and release schedule. 

Parent Network will launch on a date mutually agreed-upon between the IDOE 
and CTB in relation to the reporting turnaround schedule. 

Security 

CTB has many years of experience in handling millions of students' standardized assessment data without 
any known security breach. The website is managed by McGraw-Hill's Global Operations and Infrastructure 
Division, the group that also hosts Standard and Poors, and Platts. The data center has extensive backups and 
recovery procedures, and was operational through both September 1 1 and the New York City black out. 

Printing/Shipping 
Shipment Detail 

• Parent Network Login Letters will be printed in black and white, single impression and shipped to all 
of the corporations and nonpublic/charter/choice schools. 

Report Requirements 

Parent Network 
1 . Product 

■ Parent Network hosts PDFs of reports in addition to images 

■ Components 

» Login generation based on Pre-ID files 
» Dissemination of Logins 

i Letter to Families with login and password information 

ii School Admin codes to be able to access Parent Network 
» Application 

i Custom for IN 

a Introductory letter (landing page) 
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b IN logo 
c IN signature 

d IN content and translation guides 
e FAQ 
» Student scores 

i graphic display of the scale score 

ii achievement level 

iii Image Print PDF 

iv PDF download 

v interpretive guidance 

vi No item-level results on Parent Network site 
» Content Resources 

i Grade-banded state and national resources 

a Based on grade of students, provide 3-5 links and screenshots of web 
resources 

ii Standards-based resources 

a The description of standards in parent-friendly language, including real- 
world examples 

b Link to IN web resource that describes standards, benchmarks in detail 
c Link to IN web resource that has activities 
Hi Assessment Literacy 

a Provides links to IN released items 
b Test information 

» Image Print 

i Combined PDF for all tested subjects 

ii Includes average of 10 images of student responses (from Applied Skills portion of 

each subject test) 

iii Closely connected to links of released items 

iv Direct linkage from item-level results to item response not possible due to lack of 

PDF indexing 
» Student Reports - Providing as PDFs 

The Individual Student Reports 

1 . Multi-subject Student Reports (optimized for PDF distribution) 

2. Overall, standard-level, and item-level results 

3. Content includes introductory text, inteipretation guidance, resources 

4. Provided in 3 formats 

a. PDFs for parents on Parent Network 

b. PDFs for educators - collated by school and corporations, for electronic download on educator 
site (INORS) 

5. Student Reports printed in two separate operational runs - the first run will contain all final student data 
and the second contains any students whose scores have been ehanged during the rescore period. 
Corrected Student Reports are also posted on Parent Network and INORS. 

6. Tracking of packages provided 

Assumptions: 

• ISR PDFs will be used for both Parent Network and INORS. 
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• Files of associated barcodes with pie-codes assigned so that the barcodes are utilized for matching 
with GRT. 

Schedule Assumptions 

• Internal CTB Red Team Review of report output 

• A final review and sign-off of Student Reports by the IDOE will occur prior to release/before 
operations begin. 

o CTB will work with the IDOE to define the final Red Team Review and release schedule. 

Printing/Shipping 
Shipment Detail 

• Student Reports - One (double-sided) copy of each Student Report is created and sent to schools. 
Student Reports will be printed in black and white. (See Report Requirements for more 
specifications.) 

Report Requirements 

Individual Student Reports 

■ Student Reports - Provided as PDFs and black-and-white printed copy 
» Scores include overall, subscore and item results for 3-4 subjects 
» Format - single-subject 

i 2 versions: 

a PDFs designed for black-and-white printing 
b Black-and-white printed copy 

ii Single shell for design, no actual shells used in printing 

iii 2 page (8 Yi x 1 1) report, double-sided 
» Components 

i Front page 

a Intro letter 

b Student/School Box 

c Overall scale scores - traffic light display 
o Pass+, Pass, Did Not Pass 
o Narrative text with score and performance level 

d Performance level descriptors 

e Descriptive text integrated into letter and along bottom 
f Important general Indiana resources provided on front 

ii Back page 

a Introduction text to standard- and item-level results 
b Standard (topic) level results 

o Data reported by each standard 

■ Test item types 

■ Student IPI score (number) 

■ Passing IPI score 

■ Corporation average 

■ State average 
c Item-level results 

o Item number 
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o Standard tested 
o Page number in test booklet 
o Points Earned 
o Points possible 
d Resources - provided in all grades with some subject-specific resources; 
grade-banded 

» Delivery 

i PDFs parsed as individual files for hosting on websites 

ii Printed black-and-white copy 
■ Spanish translation 

» Translation Guide about the Individual Student Report 

Assessment Results 

a. Total subjects/grade = 2 or 3 (see chart below) 



Grade 


Reading/LA 


Math (Science 


Social Studies 


Total Subjects 


3 


X 


Xj 






2 


4 


X 


X f 


X 




3 


5 


X 


X 


X 


3 


6 


X 


X 


X 




3 


7 


X 


X I 




X 


3 


8 


X 


X f 






2 



i Performance 

ii Performance Levels ~ 3 (Did Not Pass, Pass, Pass+) 

ill Standard level results = Student IPI, Passing IPI, and State Average 
iv Data 

a Results for ISTEP+ processed at same time 

a Student results provided using the GRT 

b Unique Indiana student IDs called STN 

c Applied Skills file with barcodes assigned 

d Assessment results processed by state completion 

e See TAT schedule above for specific processing steps 
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Exhibit F 



ISTEP+ Renewal Deliverables Overview 



Planning meetings, Content/Bias Review meetings, Item selection, and 
form selection 


SeDtember 30 2012 

^^ULbl 1 1 L/C 1 ** vj ^ v 1 ^ 


Forms design, print and online production of Applied Skills Operational 
and Pilot forms 


October 3 1,2012 


AJinrt T/*\j~Kn rrv/ Paoic^rifinn ^iirv/^v/ 3nrl R f*a HinPCC f*hpf*lf DOinf 1 

V-Jnlinc I GcnnoiOFY rvccibtrdtiuii our vcy aiiu r\caunicdd L-neL.i^|jviiii J> 

Produce ancillary materials 


November 30, 2012 


Forms design, print and online production of Multiple-Choice forms 

ana \_oreuiiK mot luinis 


December 31, 2012 


Printing and fulfillment of Applied Skills Operational and pilot forms and 

Tor*- PnnrHin^tn^ \At<:* inline rf*aHinp^ ^"imnlf^rinn* Tf*<;r C* nf>rrlin#ltor 

| tiSiL V^UUI till laLUI lS.ELo t 1 II 1 1 1 C 1 Cat-ill icjo t*VJI 1 vi \f I ^ wi muvi 

trainings 


February 28, 2013 


Pfinf-inrr nnA fi ilfillmonr r\( Ml ilrinl(^-(~hnirp form 1 !' OtllitlP form 
rrintlNg d-NU lullllllllcllL KJl 1 IUIuUIg-^IUJI^c iui viMMic iu> in 

production of both operational and CoreLink pilot forms; Pickup of 
materials, Scan and score Applied Skills tests; Complete Test 
Coordinator trainings; Begin handscoring Applied Skills test 


April 30, 2013 


Complete Handscoring; Complete Site readiness checkpoints and 
Online Multiple-Choice and CoreLink Pilot tests; Pick up, Scan and 
score Multiple-Choice tests; Score and report all phases of reporting; 
Plan and conduct Science cut score setting 


lune 30 2013 


Complete Science reporting based on approved cut scores; Complete 
rescore and launch of rescore updated reports; including shipping of 
ISR and Labels to all sites; Deliver technical report 


September 30, 2013 


Planning meetings, Item and form selection, forms design, print and 
online production of Applied Skills Operational forms 


October 3 1,2013 


(""Inline Tarhnn\nav R oaicfraHnn ^nrv/^v anrl Rf*a rlinf» , ; , ; rhecknoint' 

^JIllInG 1 GCIINUIOkY P*cgtoLI dLIUM *JU\ VCJ* driu i\g<iuii IC33 v-i it^rv^ivn n. 

Produce ancillary materials 


November 30, 2013 


Forms design, print and online production of Multiple-Choice forms 
and CoreLink Pilot forms 


December Jl, 2UI J 


Printing and fulfillment of Applied Skills Operational forms and Test 
Coordinator kits; Online readiness completion; Test Coordinator 
trainings 


February 28, 2014 


Printing and fulfillment of Multiple-Choice forms; Online form 
production of both operational and CoreLink pilot forms; Pickup of 
materials, Scan and score Applied Skills tests; Complete Test 
Coordinator trainings; Begin handscoring Applied Skills test 


April 30, 2014 


Complete Handscoring; Complete Site readiness checkpoints and 
Online Multiple-Choice and CoreLink Pilot tests; Pick up, Scan and 
score Multiple-Choice tests; Score and report all phases of reporting, 
including shipping ISR and Labels to all sites 


June 30, 2014 


Complete rescore and launch of rescore updated reports; Deliver final 
technical report 


September 30, 2014 
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Exhibit G 



ISTEP+ Renewal Billing Schedule 



2012-13 


September 30, 2012 


$ 998,659 


October 31, 2012 


$ 1,125,300 


November 30, 2012 


$2,250,600 


December 31, 2012 


$ 2,925,903 


February 28, 2013 


$ 4,746,720 


April 30, 2013 


$6,388,302 


June 30, 2013 


$6,282,814 


September 30, 2013 


$1,020,616 


Total 


$25,738,914 




2013-14 


October 31, 2013 


$ 1,611,480 


November 30, 2013 


$1,701,614 


December 31, 2013 


$2,012,046 


February 28, 2014 


$ 3,577,920 


April 30, 2014 


$ 4,479,338 


June 30, 2014 


$5,251,677 


September 30, 2014 


$1,017,103 


Total 


$19,651,178 
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CTB/McG RAW-HILL LLC 
SECRETARY'S CERTIFICATE 



I, Daniel Guetta, Secretary of CTB/McG raw-Hill LLC, a Delaware limited liability company 
("CTB"), do hereby certify that the following designated employees of CTB or The McGraw-Hill 
Companies, Inc. 



Robert Bahash 
Ellen Haley 
Douglas M. Hartman 
Mark Limbach 
Joseph Micallef 
Sandor Nagy 
Anu Tummala 



are hereby authorized to make, sign and enter into bids, quotations, contracts, and agreements 
with customers for the sale of products or services by CTB, upon such terms and conditions as 
such designated persons shall deem appropriate; and, further, that such designation is intended 
to comprehend and include any specified form of authorization with respect to bids, quotations, 
contracts and agreements with customers of CTB that may be required by any statute, rule or 
regulation of the United States, any state or municipality or any subdivision or agency thereof; 
and should any such specified form of authorization be so required, that this designation shall 
be deemed to be adopted in the form so specified in the statutes, rule or specification. 



IN WITNESS WHEREOF, the undersigned has executed this Secretary's Certificate and affixed 
the seal of CTB as of the J /T"- day of February, 2012. 




DanieTGuetta 

CORPORATE SEAL Secretary 



State of New York 
County of New York 



Subscribed and sworn to before me on 
this a day of February, 201 2. 



(rfj 4 




fotary Public 



EXECUTIVE DOCUMENT SUM! 

Stats Form 41221 (R10/4-06) 

ing the EDS and the Contract process, 

OCT 26 2010 

Oiqfauidefines on the back of this form. A/" 
40 i formation. T\vC> 

OCT 2 i$m**fa ^ ap^. idoa Contracts 

4. For amendment / renewals, attach origma] contract 

5. Attach additions pages ft necessary. 




~jQ 1. Pteasereae 



. P lea aa type 



33. Will the attached document involve data processing or tdccomnnmici 


tkas syslems(s)? 


Yes: [OTor Delegate has sg 


aedorToo contract 


36. Statutory Authority (Cite applicable Indiana or Federal Codes): 








IC 20-19-3-5 









XBDS Number: 

A58-1-11SA-005 



4.974 *n//no 



2- Date prepared: 
8/16/2010 



3. CONTRACTS & LEASES 



^— Professional/Personal Services 

— Grant 

— Lease 

— Attorney 
_ MOU 

QPA 



Contract for procured Services 

Mamtenance 

License Agreement 

Amendments 

— Renewal # ~ 

Other 



FISCAL INFORMATION 



TIME PERIOD COVERED IN THIS EDS 



1 1. From (mourn, day, year): 
7/1/2010 



12. To ( month, day, year ): 
6/30/2012 



13. Method of source selection: 

Bid/Quotation Emergency 

X _ m 7-93 



Negotiated 

Special Prucuiexneut 



.Other (specify) 



AGENCY INFORMATION 


14. Name of agency: 


15. Requisition Number: 


Department of Education 




0000004290 


16. Address: DOE-ACCOUNTING 






151 WOHiO ST 






INDIANAPOLIS. IN 46204 






AGENCY CONTACT INFORMATION 


17. Name: 




18. Telephone #: 


Michete Walker 




317/232-9051 



19, E-mail ■wHrj* ^ « r ' 

iTrwaIkw®doe.in.gov 



COURIER INFORMATION 



20. Name: 
Katie DBton 



21. Telephone # : 
317-232-0569 



22. E-mail address: 

ktfOlon@doe.tn.gov 



VENDOR INFORMATION 



23 Vendor ID » 



0000269835 



4. Account Number: 


5. Account Name: 


13950-55500. 


TESTING & REMEDIATION 


6. Total amount this action: 


7. New contract total: 


$49,649,698.00 


S49.649.698.00 


8. Revenue generated this action: 


9. Revenue generated total contract 


$0.00 


$0.00 


1 0.New total amount for each fiscal year 




Year 2011 SI 9.662.862.00 




Year 2012 S2998B83B00 




Year $ 




Year $ 



24. Name: 

CTB MCGRAW-HILL LLC 



25. Telephone #: 
831-393-7401 



26. Address: 



PO BOX 2258 
CAROLSTREAMJL 60132 



27. E-mail address: Leslie JStricklandQctb.com 



28. Is the vendor registered with the Secretary of State? (Out of State 
Corporations, must be registered) X Y es No 



29. Primary Vendor M/WBE 
Minority: Yes 



Women: 



Yes 



No 
US. 



31SubVendorM/WBE 
Minority: * 
Women: 



. Yes 
X Yes 



No 
Jig. 



33. Is mere Renewal Language in 

tllC fift^TIl \f I It? 

X Yes 



No 



30.1fyes.liarhe%: 

Minority: 

Women: 



31 If yes, list the %: 
Minority: _ 
Women: 



4 6 



34. Is mere a termination for 
Convenience" clause in me 
document? X yes 



No 



37. Description of work and justification for spending money. (Please give a brief description of the scope of work included in this agreement) 

The Contractor wiO (1) provide No Child Ldt Behind (NCLB) required lea for Bmfam in grade* 3-8 throogbotfl h^arti. (Z) develop end pilot rtnns, as necessary, {3) analyze 
hem-level dxt> end crate operationi] tea forms, (4) provide the testing materials, mdinfing test booklets, mower documents, and test adnraiistratioa manuals, needed fen- successful 
test irfWKHK. (5) pnrvide *r*/i tmrmg services, and the Contractor vnll be responsible far rffl l r T, ' n fl remits >t the stadmt-. scbool-, school coiporttioQ-, nd stste-fevd and 

for provirjuig ancillary <trMi i ntf i i K wwt in die inlajiieunaQ of test resorts. 



38. Justification of vendor selection and detenninstioa of price reasonableness: 

Through comrjetraon coordmiHrrl by the radiant Department of A d m i ni st r ation, the Contractor was selected to provide NCLB-required less for students in grades 3-8._ Based on the 
proposals submitted, the price is considsird reasonable for the services provided. 



39. If this contract is aibmhted late, please "p 1 "" why: (Required If more than 30 days late.) 




4 1 . Date Approved 

10 -2\-\t> 



45. Date Approved 

afrit* 




rcceiviiig from AO 



43. Date Approved 

us-/* 



47. Date Approved 



50578-000 
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IN Department of Education 
Accounting Dept. 
151 W OHIO ST 
INDIANAPOLIS IN 46204 



IN Department of Education 
Accounting Dept. 
151 W OHIO ST 
INDIANAPOLIS IN 46204 



Requisition No. Date Required Date 
0000004290 10/18/2010 1 of 1 

Fund/Object/Centen 139507 531010 / 555000 
Dept Number 023053 
Project Number 

Requisition Number 0000004290 
Requestor M254168 Melissa Ann Covey 
Agency Number 00700 Department of Education 
Facility: 



MUST COMPLETE FOR ICPR 

- Print REQ 
Streamline Eligible 



line item 



Description 



Quantity 



UOM Unit Price 



ExtAmt 



TO ENCUMBER FUNDS. RECEIVE BY AMOUNT ONLY. 
1-1 A58111SA005 



1.0000 MON 19,662, B62. 0000 



Vendor. 



0000269835 CTB MCGRAW-HILL LLC 



19,662,862.00 



« THE CONTRACTOR WILL PROCVDE NO CHILD LEFT BEHIND REQUIRED"" 
TEST FOR STUDENTS IN GRADES 3-8 THROUGHOUT INDIANA » 

The following UN/CEFACT Unit of Measure 
Common Codes are used in this document: 
MON Month 



Requisition Total $ 19,662,862.00 





I certify that the itemfs] requested Is fare] necessary for the o Deration of this State Aaencv. 






Printed Name of Agency Head or Authorized Employee 


Authorized Signature 



State Form 21301 (Rsvised 7/02) Approved by Stats Board of Accounts - 2002 



PROFESSIONAL SERVICES CONTR 



EDS#A58-l-HSA-005 

This Contract ("this Contract"), entered into by and between the Indiana Department of Education (the 
"State") and CTB McGraw-Hill (the "Contractor"), is executed pursuant to the terms and conditions set 
forth herein. In consideration of those mutual undertakings and covenants, the parties agree as follows: 

1. Duties of Contractor 

Details of the duties are described in Exhibit A and Exhibit B attached herein and incorporated by 
reference. This represents die State's exercise of its option for the first two-year extension authorized 
under RFP #7-93, Component 3, Grades 3-8 End of Year Summative Assessments, as further defined 
herein. RFP #7-93 is Exhibit D attached herein and incorporated by reference. 

The Contractor shall provide the following services relative to this Contract: 

A. provide NCLB- required test for students in grades 3-8 throughout Indiana, 

B. develop and pilot items, as necessary, 

C. analyze item-level data and create operational test forms, 

D. provide testing materials, including test booklets, answer documents, and test administration 
manuals, needed for successful test sessions, 

E. provide scanning and scoring services, and 

F. be responsible for returning results at the student-, school-, school corporation-, and state-level 
and for providing ancillary documents to assist in the interpretation of test results. 

2. Consideration 

The Contractor will be paid in accordance with the pricing and payment schedule described in Exhibit C 
attached and herein incorporated by reference for performing the duties set forth above. Total 
remuneration under this Contract shall not exceed Forty-nine Million Six Hundred Forty-nine Thousand 
Six Hundred Ninety-eight Dollars ($49,649,698). 

3. Term 

This Contract shall be effective for a period of two years. It shall commence on July 1, 2010 and shall 
remain in effect through June 30, 2012. 

4. Access to Records 

Hie Contractor and its subcontractors, if any, shall maintain all books, documents, papers, accounting 
records, and other evidence pertaining to all costs incurred under this Contract They shall make such 
materials available at their respective offices at all reasonable times during this Contract, and for three (3) 
years from the date of final payment under this Contract, for inspection by die State or its authorized 
designees. Copies shall be furnished at no cost to the State if requested. 

5. Assignment; Successors 

The Contractor binds its successors and assignees to all the terms and conditions of this Contract. The 
Contractor shall not assign or subcontract the whole or any part of this Contract without the State's prior 
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written consent The Contractor may assign its right to receive payments to such third parties as the 
Contractor may desire without the prior written consent of the State, provided that the Contractor gives 
written notice (including evidence of such assignment) to die State thirty (30) days in advance of any 
payment so assigned. The assignment shall cover all unpaid amounts under this Contract and shall not be 
made to more than one party. 

6. Audits 

The Contractor acknowledges that it may be required to submit to an audit of funds paid through this 
Contract Any such audit shall be conducted in accordance with IC 5-1 1-1, etseq., and audit guidelines 
specified by the State. 

7. Authority to Bind Contractor 

The signatory for the Contractor represents that he/she has been duly authorized to execute this Contract 
on behalf of the Contractor and has obtained all necessary or applicable approvals to make this Contract 
fully binding upon the Contractor when his/her signature is affixed, and accepted by the State. 

8. Changes in Work 

The Contractor shall not commence any additional work or change the scope of the work until authorized 
in writing by the State. The Contractor shall make no claim for additional compensation in the absence of 
a prior written approval and amendment executed by all signatories hereto. This Contract may only be 
amended, supplemented or modified by a written document executed in the same manner as this Contract. 

9. Compliance with Laws 

A. The Contractor shall comply with all applicable federal, state, and local laws, rules, regulations, and 
ordinances, and all provisions required thereby to be included herein are hereby incorporated by 
reference. The enactment or modification of any applicable state or federal statute or the 
promulgation of rules or regulations thereunder after execution of this Contract shall be reviewed by 
the State and die Contractor to determine whether the provisions of mis Contract require formal 
modification. 

B. The Contractor and its agents shall abide by all ethical requirements that apply to persons who have a 
business relationship with the State as set forth in IC 4-2-6, et seq., IC 4-2-7, et seq^ the regulations 
promulgated thereunder, and Executive Order 04-08, dated April 27, 2004. If the Contractor is not 
familiar with these ethical requirements, the Contractor should refer any questions to the Indiana State 
Ethics Commission, or visit the Indiana State Ernies Commission website at 

http7/www. in.gov/ethics/ . If the Contractor or its agents violate any applicable ethical standards, the 
State may, in its sole discretion, terminate this Contract immediately upon notice to the Contractor. 
In addition, the Contractor may be subject to penalties under IC §§ 4-2-6, 4-2-7, 35-44-1-3, and under 
any other applicable laws. 

C. Hie Contractor certifies by entering into this Contract that neither it nor its principals) is presently in 
arrears in payment of taxes, permit fees or other statutory, regulatory or judicially required payments 
to the State of Indiana. The Contractor agrees that any payments currently due to the State of Indiana 

' may be withheld from payments due to the Contractor. Additionally, further work or payments may 
be withheld, delayed, or denied and/or this Contract suspended until the Contractor is current in its 
payments and has submitted proof of such payment to the State. 
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D. The Contractor warrants that it has no current, pending or outstanding criminal, civil, or enforcement 
actions initiated by the State, and agrees that it will immediately notify the State of any such actions. 
During the term of such actions, the Contractor agrees that the State may delay, withhold, or deny 
work under any supplement, amendment, change order or other contractual device issued pursuant to 
- this Contract. 

B. If a valid dispute exists as to the Contractor's liability or guilt in any action initiated by the State or its 
agencies, and the State decides to delay, withhold, or deny work to the Contractor, the Contractor 
may request that it be allowed to continue, or receive work, without delay. The Contractor must 
submit, in writing, a request for review to the Indiana Department of Administration (IDOA) 
following the procedures for disputes outlined herein. A determination by IDOA shall be binding on 
the parties. Any payments that the State may delay, withhold, deny, or apply under this section shall 
not be subject to penalty or interest, except as permitted by IC 5-17-5. 

F. The Contractor warrants that the Contractor and its subcontractors, if any, shall obtain and maintain 
all required permits, licenses, registrations, and approvals, and shall comply with all health, safety, 
and environmental statutes, rules, or regulations in the performance of work activities for the State. 
Failure to do so may be deemed a material breach of this Contract and grounds for immediate 
termination and denial of further work with the State. 

G. The Contractor affirms mat, if it is an entity described in IC Title 23, it is properly registered and 
owes no outstanding reports to the Indiana Secretary of State. 

H. As required by IC 5-22-3-7: 

( 1 ) The Contractor and any principals of the Contractor certify that: 

(A) the Contractor, except for de minimis and nonsystematic violations, has not violated the terms 
of: 

(i) IC 24-4.7 [Telephone Solicitation Of Consumers]; 

(ii) IC 24-5- 1 2 [Telephone Solicitations]; or 

(tii)IC 24-5-14 [Regulation of Automatic Dialing Machines]; 

in the previous three hundred sixty-five (365) days, even if IC 24-4.7 is preempted by federal 
law; and 

(B) the Contractor will not violate the terms of IC 24-4.7 for the duration of the Contract, even if 
IC 24-4.7 is preempted by federal law. 

. (2) The Contractor and any principals of the Contractor certify that an affiliate or principal of the 
Contractor and any agent acting on behalf of the Contractor or on behalf of an affiliate or 
principal of the Contractor 

(A) except for de minimis and nonsystematic violations, has not violated the terms of IC 24-4.7 in 
the previous three hundred sixty-five (365) days, even if IC 24-4.7 is preempted by federal 
law; and 

(B) will not violate the terms of IC 24-4.7 for the duration of the Contract, even if IC 24-4.7 is 
preempted by federal law. 

10. Condition of Payment 

AH services provided by the Contractor under this Contract must be performed to the State's reasonable 
satisfaction, as determined at the discretion of the undersigned State representative and in accordance 
with all applicable federal, state, local laws, ordinances, rules and regulations. The State shall not be 
required to pay for work found to be unsatisfactory, inconsistent with mis Contract or performed in 
violation of and federal, state or local statute, ordinance, rule or regulation. 
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11. Confidentiality of State Information 

The Contractor understands and agrees that data, materials, and information disclosed to the Contractor 
may contain confidential and protected information. The Contractor covenants that data, material, and 
information gathered, based upon or disclosed to the Contractor for the purpose of mis Contract, will not 
be disclosed to or discussed with third parties without the prior written consent of the State. 

The parties acknowledge that the services to be performed by Contractor for the State under this contract 
may require or allow access to data, materials, and information containing Social Security numbers 
maintained by the State in its computer system or other records. In addition to the covenant made above 
in this section and pursuant to 10 IAC 5-3-1(4), the Contractor and the State agree to comply with the 
provisions of IC 4-1-10 and IC 4-1-11. If any Social Security numbers) is/are disclosed by Contractor, 
Contractor agrees to pay the cost of the notice of disclosure of a breach of the security of the system in 
addition to any other claims and expenses for which it is liable under the terms of this contract. 

12. Continuity of Services 

A. The Contractor recognizes that the servicefs) to be performed under this Contract are vital to die State 
and must be continued without interruption and that, upon Contract expiration, a successor, either the 
State or another contractor, may continue them. The Contractor agrees to: 

1. Furnish phase-in training; and 

2. Exercise its best efforts and cooperation to effect an orderly and efficient transition to a successor. 

B. The Contractor shall, upon the State's written notice: 

1 . Furnish phase-in, phase-out services for up to sixty (60) days after this Contract expires; and 

2. Negotiate in good faith a plan with a successor to determine the nature and extent of phase- in, 
phase-out services required. 

The plan shall specify a training program and a date for transferring responsibilities for each 
division of work described in the plan, and shall be subject to the State's approval. The 
Contractor shall provide sufficient experienced personnel during the phase-in, phase-out 
period to ensure that the services called for by this Contract are maintained at the required 
level of proficiency. 

C. The Contractor shall allow as many personnel as practicable to remain on the job to help the 
successor maintain the continuity and consistency of the services required by this Contract The 
Contractor also shall disclose necessary personnel records and allow the successor to conduct on-site 
interviews with these employees. If selected employees are agreeable to the change, the Contractor 
shall release them at a mutually agreeable date and negotiate transfer of their earned fringe benefits to 
the successor. 

D. The Contractor shall be reimbursed for all reasonable phase-in, phase-out costs (i.e., costs incurred 
within the agreed period after contract expiration that result from phase-in, phase-out operations). 

13. Debarment and Suspension 

A. The Contractor certifies by entering into this Contract that neither it nor its principals nor any of its 
subcontractors are presently debarred, suspended, proposed for debarment, declared ineligible or 
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voluntarily excluded from entering into this Contract by any federal agency or by any department, 
agency or political subdivision of the State of Indiana. The term "principal" for purposes of this 
~ Contract means an officer, director, owner, partner, key employee or other person with primary 
management or supervisory responsibilities, or a person who has a critical influence on or substantive 
control over the operations of the Contractor. 

B. The Contractor certifies that it has verified the state and federal suspension and debarment status for 
all subcontractors receiving funds under this Contract and shall be solely responsible for any 
recoupment, penalties or costs that might arise from use of a suspended or debarred subcontractor. 
The Contractor shall immediately notify die State if any subcontractor becomes debarred or 
suspended, and shall, at the State's request, take all steps required by the State to terminate its 
contractual relationship with the subcontractor for work to be performed under this Contract. 

14. Default by State 

If the State, sixty (60) days after receipt of written notice, fails to correct or cure any material breach of 
this Contract, the Contractor may cancel and terminate this Contract and institute the appropriate 
measures to collect monies due up to and including the date of termination. 

15. Disputes 

A. Should any disputes arise with respect to this Contract, the Contractor and the State agree to act 
immediately to resolve such disputes. Time is of the essence in the resolution of disputes. 

B. The Contractor agrees mat, the existence of a dispute notwithstanding, it will continue without delay 
to carry out all of its responsibilities under this Contract that are not affected by the dispute. Should 
the Contractor fail to continue to perform its responsibilities regarding all non-disputed work, without 
delay, any additional costs incurred by the State or the Contractor as a result of such failure to 
proceed shall be borne by the Contractor, and the Contractor shall make no claim against the State for 
such costs. * 

C. If a party to the Contract is not satisfied with the progress toward resolving a dispute, the party must 
notify in writing the other party of this dissatisfaction. Upon written notice, the parries have ten ( 1 0) 
working days, unless the parties mutually agree to extend this period, following the notification to 
resolve the dispute. If the dispute is not resolved within ten (10) working days, a dissatisfied party 
will submit the dispute in writing according to the following procedure: 

1 . The parties agree to resolve such matters through submission in writing of their dispute to the 
Commissioner of the Indiana Department of Administration. The Commissioner shall reduce a 
decision to writing and mail or otherwise furnish a copy thereof to the Contractor and the State 
within ten (10) working days after presentation of such dispute for action. The presentation may 
include a period of negotiations, clarifications, and mediation sessions and will not terminate until 
die Commissioner or one of the parties concludes that the presentation period is over. The 
Commissioner's decision shall be final and conclusive unless either party mails or otherwise 
furnishes to the Commissioner, within ten (10) working days after receipt of the Commissioner's 
decision, a written appeal. Within ten (10) working days of receipt by the Commissioner of a . 
written request for appeal, the decision may be reconsidered. If no reconsideration is provided 
within ten (10) working days, the parties may mutually agree to submit the dispute to arbitration 
or mediation for a determination. If a party is not satisfied with the Commissioner's ultimate 
decision, the dissatisfied party may submit the dispute to an Indiana court of competent 
jurisdiction. 
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2. The State may withhold payments on disputed items pending resolution of the dispute. The 
unintentional nonpayment by the State to the Contractor of one or more invoices not in dispute in 
accordance with the terms of this Contract will not be cause for the Contractor to terminate this 
Contract, and the Contractor may bring suit to collect these amounts without following the 
disputes procedure contained herein. 

16. Drug-Free Workplace Certification 

The Contractor hereby covenants and agrees to make a good faith effort to provide and maintain a drug- 
free workplace. The Contractor will give written notice to the State within ten (10) days after receiving 
actual notice that the Contractor or an employee of the Contractor in the State of Indiana has been 
convicted of a criminal drug violation occurring in the workplace. False certification or violation of this 
certification may result in sanctions including, but not limited to, suspension of contract payments, 
termination of this Contract and/or debarment of contracting opportunities with the State for up to three 
(3) years. 

In addition to the provisions of the above paragraphs, if the total contract amount set forth in mis Contract 
is in excess of $25,000.00, the Contractor hereby further agrees mat this Contract is expressly subject to 
the terms, conditions, and representations of the following certification: 

This certification is required by Executive Order No. 90-5, April 1 2, 1 990, issued by the Governor of 
Indiana. No award of a contract shall be made, and no contract, purchase order or agreement, the total 
amount of which exceeds $25,000.00, shall be valid, unless and until this certification has been fully 
executed by the Contractor and made a part of the contract or agreement as part of the contract 
documents. .- 

The Contractor certifies and agrees that it will provide a drug-free workplace by: 

A. Publishing and providing to all of its employees a statement notifying them that the unlawful 
manufacture, distribution, dispensing, possession or use of a controlled substance is prohibited in the 
Contractor's workplace, and specifying the actions that will be taken against employees for violations 
of such prohibition; 

B. Establishing a drug-free awareness program to inform its employees of ( 1 ) the dangers of drug abuse 
in the workplace; (2) the Contractor's policy of maintaining a drug-free workplace; (3) any available 
drug counseling, rehabilitation and employee assistance programs; and (4) the penalties that may be 
imposed upon an employee for drug abuse violations occurring in the workplace; 

C. Notifying all employees in the statement required by subparagraph (A) above mat as a condition of 
continued employment, the employee will ( 1 ) abide by the terms of the statement; and (2) notify the 
Contractor of any criminal drug statute conviction for a violation occurring in the workplace no later 
than five (5) days after such conviction; 

D. Notifying the State in writing within ten (10) days after receiving notice from an employee under 
subdivision (C)(2) above, or otherwise receiving actual notice of such conviction; 

E. . Within thirty (30) days after receiving notice under subdivision (C)(2) above of a conviction, 

imposing the following sanctions or remedial measures on any employee who is convicted of drug 
abuse violations occurring in the workplace: (1) taking appropriate personnel action against the 
employee, up to and including termination; or (2) requiring such employee to satisfactorily participate 
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tit a drug abuse assistance or rehabilitation program approved for such purposes by a federal, state or 
local health, law enforcement, or other appropriate agency; and 

F. Making a good faith effort to maintain a drug-free workplace through the implementation of 
subparagraphs (A) through (E) above. 

17. Employment Option 

If the State determines that it would be in the State's best interest to hire an employee of the Contractor, 
the Contractor will release the selected employee from any non-compete agreements mat may be in effect. 
This release will be at no cost to the State or the employee. 

18. Force Majeure 

In the event that either parry is unable to perform any of its obligations under mis Contract or to enjoy any 
of its benefits because of natural disaster or decrees of governmental bodies not the fault of the affected 
party (hereinafter referred to as a "Force Majeure Event"), the party who has been so affected shall 
immediately give notice to the other party and shall do everything possible to resume performance. Upon 
receipt of such notice, all obligations under mis Contract shall be immediately suspended. If the period of 
nonperformance exceeds thirty (30) days from the receipt of notice of the Force Majeure Event, the parry 
whose ability to perform has not been so affected may, by giving written notice, terminate this Contract. 

19. Funding Cancellation 

When the Director of the State Budget Agency makes a written determination that funds are not 
appropriated or otherwise available to support continuation of performance of this Contract, this Contract 
shall be canceled. A determination by the Director of SBA that funds are not appropriated or otherwise 
available to support continuation of performance shall be final and conclusive. 

20. Governing Laws 

This Contract shall be construed in accordance with and governed by the laws of the State of Indiana, and 
suit, if any, must be brought in the State of Indiana. 

21. Indemnification 

The Contractor agrees to indemnify, defend, and hold harmless the State, its agents, officials, and 
employees from all claims and suits including court costs, attorney's fees, and other expenses caused by 
any act or omission of the Contractor and/or its subcontractors, if any, in the performance of this Contract. 
The State shall not provide such indemnification to the Contractor. 

22. Independent Contractor 

Bom parties hereto, in the performance of this Contract, shall act in an individual capacity and not as 
agents, employees, partners, joint venturers or associates of one another. The employees or agents of one 
parry shall not be deemed or construed to be the employees or agents of the other party for any purposes 
whatsoever. Neither party will assume liability for any injury (including death) to any persons, or damage 
to any property, arising out of the acts or omissions of the agents, employees or subcontractors of the 
other party. 
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The Contractor shall be responsible for providing all necessary unemployment and workers* 
compensation insurance for the Contractor's employees. 

23. Information Technology Enterprise Architecture Requirements 

If the Contractor provides any information technology related products or services to the State, the 
Contractor shall comply with all IOT standards, policies and guidelines, which are online at 
http://iotin.gov/architecture/. The Contractor specifically agrees that all hardware, software and services 
provided to or purchased by the State shall be compatible with the principles and goals contained in the 
electronic and information technology accessibility standards adopted under Section 508 of the Federal 
Rehabilitation Act of 1973 (29 U.S.C. 794d) and IC 4-13.1-3. Any deviation from these architecture 
requirements must be approved in writing by IOT in advance. The State may terminate this Contract for 
default if the Contractor mils to cure a breach of this provision within a reasonable time. 

24. Insurance (This clause deleted by agreement of the parties) 

25. Key Person(s) (This clause deleted by agreement of the parties) 

26. licensing Standards 

The Contractor, its employees and subcontractors shall comply with all applicable licensing standards, 
certification standards, accrediting standards and any other laws, rules, or regulations governing services 
to be provided by the Contractor pursuant to this Contract Hie State will not pay the Contractor for any 
services performed when the Contractor, res employees or subcontractors are not in compliance with such 
applicable standards, laws, rules, or regulations. If any license, certification or accreditation expires or is 
revoked, or any disciplinary action is taken against ah applicable license, certification, or accreditation, 
the Contractor shall notify the State immediately and the State, at its option, may immediately terminate 
this Contract 

27. Merger & Modification 

This Contract constitutes the entire agreement between the parties. No understandings, agreements, or 
representations, oral or written, not specified within this Contract will be valid provisions of this Contract. 
This Contract may not be modified, supplemented, or amended, except by written agreement signed by all 
necessary parties. 

28. Minority and Women's Business Enterprises Compliance 

The Contractor agrees to comply fully with the provisions of 25 IAC S and the Subcontractor 
Commitment submitted to the State. No changes may be made to the commitment without the written 
approval of the Minority and Women's Enterprises Division of IDOA. 

The following MBE's and WBE's listed on the Minority and Women's Business Enterprises Division 
directory of certified firms will be participating in this Contract. 

■ COMPANY - 
MBE/WBE PHONE NAME SCOPE OF PRO DUCTS ami/or SERVICES UTILIZATION DATE AMOUNT 

All Pack 

mL 317/378-7225 Specialists Provide corrugated boxes 7/10 to 3/12 S10 Q. O0P 

Training Services including technology 
WBE 260/434-0990 BrilientLLC services, onsitc training and site survey 12/10 to 4/12 S184J2Q 
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Langham 

WBE 317/872-4490 Log&ks Transportation and logistics 9/10 to 4/12 11.500.000 



Professional staffing for contract 
MBE 214/550^124 Dolovit employees 9/10 to 5/12 $2300.000 

Interactive Printing large print books, 

WBE Printing* and manipulativcs, paper bands, stack cards 

8567486-0066 P ackagin g and other materials 5/10 to 1/12 1359.090 

'(Certification in process) 

The Contractor agrees to submit a copy of the agreement entered into between the Contractor and each 
MBE/WBE subcontractor where the State took the selection of the MBE/WBE by the Contractor into 
consideration when issuing the procurement award. The copy of the agreement must be submitted to the 
MWBE Division in IDOA within ninety (90) days of the execution of the contract between the Contractor 
and the State. The Contractor also agrees to send all amendments, changes, and terminations to these 
agreements to the MWBE Division in IDOA within ninety (90) days of their execution. Failure to 
provide a copy of the agreement or subsequent amendment, change, and termination may result in 
exclusion from future State procurements. If the Contractor is not excluded from future procurements, 
the actions or inactions of the Contractor with regard to the above will be taken into account in all phases 
and scoring in future procurements. In addition, the Contractor must obtain the approval of the Division 
before changing any MBE/WBE participation plan submitted in connection with this Contract. 

29. Nondiscrimination 

This covenant is enacted pursuant to the Indiana Civil Rights Law, specifically including 1C 22-9-1-10, 
and in keeping with the purposes of the Civil Rights Act of 1964, the Age Discrimination in Employment 
Act, and the Americans with Disabilities Act. Breach of this covenant may be regarded as a material 
breach of this Contract, but nothing in this covenant shall be construed to imply or establish an 
employment relationship between the State and any applicant or employee of the Contractor or any 
subcontractor. 

Pursuant to the Indiana Civil Rights Law, specifically including IC 22-9-1-10, and in keeping with the 
purposes of die federal Civil Rights Act of 1964, the Age Discrimination in Employment Act, and the 
Americans with Disabilities Act, the Contractor covenants that it shall not discriminate against any 
employee or applicant for employment relating to this Contract with respect to the hire, tenure, terms, 
conditions or privileges of employment or any matter directly or indirectly related to employment, , 
because of the employee or applicant's: race, color, national origin, religion, sex, age, disability, ancestry, 
status as a veteran, or any other characteristic protected by federal, state, or local law ("Protected 
Characteristics"). Furthermore, Contractor certifies compliance with applicable federal laws, regulations, 
and executive orders prohibiting discrimination based on the Protected Characteristics in the provision of 
services. 

30. Notice to Parties 

Whenever any notice, statement or other communication is required under this Contract, it shall be sent to 
die following addresses, unless otherwise specifically advised. 

A. Notices to the State shall be sent to: 
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Michele Walker 
Director of Student Assessment 
Indiana Department of Education 
151 W.Ohio Street 
Indianapolis, IN 46204 

B. Notices to the Contractor shall be sent to: 

Leslie Strickland 
Program Manager 
CTB McGraw-Hill 
20 Ryan Ranch Road 
Monterey, CA 93940 

. C. As required by IC 4-13-2-14.8, payments to the Contractor shall be made via electronic funds transfer 
in accordance with instructions filed by the Contractor with the Indiana Auditor of State. 

31. Order of Precedence; Incorporation by Reference 

Any inconsistency or ambiguity in this Contract shall be resolved by giving precedence in the following 
order (1) This Contract, (2) attachments prepared by the State, (3) RFP#7-93, (4) Contractor's response 
to RFP#7-93, and (5) attachments prepared by the Contractor. All of the foregoing are incorporated fully 
by reference. All attachments, and all documents referred to in this paragraph are hereby incorporated 
fully by reference. 

32. Ownership of Documents and Materials 

All documents, records, programs, data, film, tape, articles, memoranda, and other materials not 
developed or licensed by the Contractor prior to execution of this Contract, but specifically developed 
under this Contract shall be considered work for hire" and the Contractor transfers any ownership claim 
to the State and all such materials will be the property of the State. Use of these materials, other than 
related to contract performance by the Contractor, without the prior written consent of the State, is 
prohibited. During the performance of mis Contract, the Contractor shall be responsible for any loss of or 
damage to these materials developed for or supplied by the State and used to develop or assist in the 
services provided while the materials are in the possession of the Contractor. Any loss or damage thereto 
shall be restored at the Contractor's expense. The Contractor shall provide the State full, immediate, and 
unrestricted access to the work product during the term of this Contract. 

33. Payments 

A. All payments shall be made in arrears in conformance with State fiscal policies and procedures and, 
as required by IC 4-13-2-14.8, by electronic funds transfer to the financial institution designated by 
the Contractor in writing unless a specific waiver has been obtained from the Indiana Auditor of 
State. No payments will be made in advance of receipt of the goods or services that are the subject of 
this Contract except as permitted by IC 4- 13-2-20. 

B. The State Budget Agency and the Contractor acknowledge mat Contractor is being paid in advance 
for the maintenance of equipment and / or software. Pursuant to IC 4- 1 3-2-20(bX 1 4), Contractor 
agrees mat if it fails to perform the maintenance required under mis Contract, upon receipt of written 
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notice from the State, it shall promptly refund the consideration paid, pro-rated through the date of 
non-performance. 

C. All accounts will be closed sixty (60) days after the Expiration Date of this Contract Agreement. Any 
- invoice submitted after sixty (60) days will not be reimbursed by the State. 

D. The Contractor agrees to abide by the Contract Budget; except that, subject to written approval of 
the State Project Director, budget line hem amounts may be transferred to other Contract budget 

line items. The maximum amount that may be transferred from any one Contract budget line item is 
equal to ten percent (10%) of the original budget line item amount 

34. Penal ties/Interest/Attorney's Fees 

The State will in good faith perform its required obligations hereunder and does not agree to pay any 
penalties, liquidated damages, interest or attorney's fees, except as permitted by Indiana law, in part, IC 
5-17-5, IC 34-54-8, and IC 34-13-1. 

Notwithstanding the provisions contained in IC 5-1 7-5, any liability resulting from the State's failure to 
make prompt payment shall be based solely on the amount of funding originating from the State and shall 
not be based on funding from federal or other sources. 

35. Progress Reports 

The Contractor shall submit progress reports to the State upon request. "The report shall be oral, unless the 
State, upon receipt of the oral report, should deem it necessary to have it in written form. The progress 
reports shall serve the purpose of assuring the State that work is progressing in line with the schedule, and 
that completion can be reasonably assured on the scheduled date. 

36. Renewal Option 

This Contract may be renewed under the same terms and conditions, subject to the approval of the 
Commissioner of the Department of Administration and the State Budget Director in compliance with IC 
5-22- 1 7-4. The term of the renewed contract may not be longer than the term of the original contract. 

37. Security and Privacy of Health Information 

The Contractor agrees to comply with all requirements of the Health Insurance Portability and 
Accountability Act of 1996 (HIPAA) in all activities related to this Contract, to maintain compliance 
throughout the lite of mis Contract, to operate any systems used to fulfill the requirements of this 
Contract in full compliance with HIPAA and to take no action which adversely affects the State's HIPAA 
compliance. 

The parties acknowledge that the Department of Health and Human Services has issued the Final Rule, as 
amended from time to time, on the Standards for Privacy of Individually Identifiable Hearth Information, 
as required by HIPAA. To the extent required by the provisions of HIPAA and regulations promulgated 
thereunder, the Contractor covenants that it will appropriately safeguard Protected Health Information 
(PHI), as defined by the regulations, which is made available to or obtained by the Contractor in the 
course of its work under this Contract The Contractor agrees to comply with applicable requirements of 
law relating to PHI with respect to any task or other activity it performs for the State as required by the 
final regulations. , 
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38. Severability 

Hie invalidity of any section, subsection, clause or provision of this Contract shall not affect the validity 
of the remaining sections, subsections, clauses or provisions of this Contract. 

39. Substantial Performance 

This Contract shall be deemed to be substantially performed only when fully performed according to its 
terms and conditions and any written amendments or supplements. 

40. Taxes 

The State is exempt from most state and local taxes and many federal taxes. The State will not be 
responsible for any taxes levied on the Contractor as a result of mis Contract. 

41. Termination for Convenience 

This Contract may be terminated, in whole or in part, by the State, which shall include and is not limited 
to the Indiana Department of Administration and the State Budget Agency whenever, for any reason, the 
State determines that such termination is in its best interest Termination of services shall be effected by 
delivery to the Contractor of a Termination Notice at least thirty (30) days prior to the termination 
effective date, specifying the extent to which performance of services under such termination becomes 
effective. The Contractor shall be compensated for services properly rendered prior to the effective date 
of termination. The State will not be liable for services performed after the effective date of termination. 
The Contractor shall be compensated for services herein provided but in no case shall total payment made 
to the Contractor exceed the original contract price or shall any price increase be allowed on individual 
line herns if canceled only in part prior to the original termination date. For the purposes of this 
paragraph, the parties stipulate and agree that the Indiana Department of Administration shall be deemed 
to be a party to this agreement with authority to terminate the same for convenience when such 
termination is determined by the Commissioner of IDOA to be in the best interests of the State. 

42. Termination for Default 

A. With the provision of thirty (30) days notice to the Contractor, the State may terminate this Contract 
in whole or in part if the Contractor fails to: 

1 . Correct or cure any breach of this Contract; the time to correct or cure the breach may be 
extended beyond thirty (30) days if the State determines progress is being made and the extension 
is agreed to by the parties; 

2. Deliver the supplies or perform the services within the time specified in this Contract or any 
extension; 

3. Make progress so as to endanger performance of this Contract; or 

4. Perform any. of the other provisions of this Contract 

B. If the State terminates this Contract in whole or in part, it may acquire, under the terms and in the 
manner the State considers appropriate, supplies or services similar to those terminated, and the 
Contractor will be liable to the State for any excess costs for those supplies or services. However, the 
Contractor shall continue the work not terminated. 
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C. The State shall pay the contract price for completed supplies delivered and services accepted. The 
Contractor and die State shall agree on the amount of payment for manufacturing materials delivered 
and accepted and for die protection and preservation of die property. Failure to agree will be a 
dispute under the Disputes clause. The State may withhold from these amounts any sum the State 
determines to be necessary to protect the State against loss because of outstanding liens or claims of 
former lien holders. 

D. The rights and remedies of the State in this clause are in addition to any other rights and remedies 
provided by law or equity or under this Contract. 

43. Travel 

No expenses for travel will be reimbursed unless specifically permitted under the scope of services or 
consideration provisions. Expenditures made by the Contractor for travel will be reimbursed at the 
current rate paid by die State and in accordance with die State Travel Policies and Procedures as specified 
in die current Financial Management Circular. Out-of-state travel requests must be reviewed by the State 
for availability of funds and for appropriateness per Circular guidelines. 

44. Waiver of Rights 

No right conferred on either party under this Contract shall be deemed waived, and no breach of this 
Contract excused, unless such waiver is in writing and signed by the party claimed to have waived such 
right. Neither the State's review, approval or acceptance of, nor payment for, the services required under 
this Contract shall be construed to operate as a waiver of any rights under this Contract or of any cause of 
action arising out of the performance of this Contract, and the Contractor shall be and remain liable to the 
State in accordance with applicable law for all damages to die State caused by die Contractor's negligent 
performance of any of the services furnished under this Contract 

45. Work Standards 

The Contractor shall execute its responsibilities by following and applying at all times the highest 
professional and technical guidelines and standards. If the State becomes dissatisfied with the work 
product of or the working relationship with those individuals assigned to work on this Contract, the State 
may request in writing the replacement of any or all such individuals, and the Contractor shall grant such 
request 

46. State Boilerplate Affirmation Clause 

I swear or affirm under die penalties of perjury that I have not altered, modified, changed or deleted the 
State's Boilerplate contract clauses (as contained in the 2010 OAG/ IDOA Professional Services Contract 
Manual) in any way except for die following clauses which are named below: 

2. Consideration (First sentence modified to reference to pricing and payment schedule 
in Exhibit C). 

24. Insurance (Entire clause deleted by agreement of the parties) ' 

25. Key Person(s) (Entire clause deleted by agreement of die parties) 

33. Payments (Items C, billing period added and Item D, budget transfer information added) 

47. Liquidated Damages (Added) 

General timelines for deliverables and services under this Contract are set form in Exhibit A (Work 
Scope) hereto. 
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A. It is hereby agreed that, without limitation to other rights and remedies of the State under this 
Contract, the State shall be entitled to liquidated damages for the Contractor's failure to deliver, 

. pursuant to the provisions of the Contract, any of the following five (S) deliverables: 

(1) paper testing materials delivery; 

(2) print reports delivery, 

(3) "INORS" summary report delivery; 

(4) "Parent Network*' report delivery; and 

(5) "OAS" online testing delivery, which is defined as uninterrupted CTB system 
availability every school day from 7:00 a.m. to 6:00 pjn. Eastern time for the two weeks 
prior to the testing window and during the entire testing window 

in accordance with the Program Milestones and Critical Dates for each year of the Contract as 
specified in Exhibit A hereto. In the event of the Contractor's failure, liquidated damages shall 
begin to accrue the calendar day of such failure of Contractor to perform. The State shall be 
. entitled to liquidated damages in the amount of $50,000 per day for each of the first five (5) 
calendar days of the Contractor's failure to deliver, $150,000 per day for each of the next five (5) 
calendar days (days 6-10) and $250,000 per day for each calendar day thereafter until such time 
mat the Contractor cures the failure to deliver. The liquidated damages for any single delivery 
failure shall not exceed 1 0% of the current contract amount 

B. It is hereby agreed further that, without limitation to other rights and remedies of the State under 
this Contract, the State shall be entitled to liquidated damages for the Contractor's failure to 
deliver specified deliverables or complete specified tasks pursuant to the Contract and not 
otherwise specified in paragraph A above, in accordance with the Program Milestones and 
Critical Dates for each year of the Contract as specified in Attachments P and G within Exhibit A 
hereto. The State shall provide the Contractor with written notice of the Contractor's failure to 
perform, and the Contractor shall have five (5) calendar days from receipt of the written notice to 
cure die failure without penalty. In the event that the Contractor fails to cure within the prescribed 
time period, liquidated damages shall begin to accrue, and the State shall be entitled to liquidated 
damages in the amount of $25,000 per day for each of the first five (5) calendar days after the 
time period for curing without penalty has elapsed and $ 1 00,000 per day for each calendar day 
thereafter until such time that the Contractor cures the failure to deliver or otherwise perform. 
The liquidated damages for any single delivery or performance failure shall not exceed 1 0% of 
the current contract amount. 

C. It is agreed that liquidated damages will not accrue against the Contractor for failure to meet an 
original Program Milestone or Critical Date if, prior to the specific Program Milestone or Critical 
Date, the parties hereto mutually agree upon a change of date for a Program Milestone or Critical 
Date, and an amendment incorporating the change of date is executed by the parties to the 
Contract. However, liquidated damages as described in paragraphs A and B above are applicable 
should the Contractor rail to perform in accordance with the amended Program Milestones and 
Critical Dates deadlines). 

D. It is agreed that, should the Contractor be prevented from timely performing its obligations 
hereunder by a force majeure event, including the State's acts or failure to act in its sovereign 
and contractual capacity such as a change in the scope of work or the failure to provide 
required approvals, the Contractor's time to perform shall be extended for the period of delay 
resulting from such events) as determined by the State. 
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Non-Collusion and Acceptance 



The undersigned attests, subject to die penalties for perjury, that he/she is the Contractor, or that he/she is the 
properly authorized representative, agent, member or officer of the Contractor, that he/she has not; nor has 
any other member, employee, representative, agent or officer of the Contractor, directly or indirectly, to the 
best of the undersigned's knowledge, entered into or offered to enter into any combination, collusion or 
agreement to receive or pay, and that he/she has not received or paid, any sum of money or other 
consideration for the execution of this Contract other than mat which appears upon the face of this Contract 

In Witness Whereof, Contractor and the State have, through their duly authorized representatives, 
entered into this Contract Hie parties, having read and understood the foregoing terms of mis Contract, 
do by their respective signatures dated below hereby agree to die terms thereof. 



CTB McGraw-Hill: 



By: 

Printed Name: Malf-T 

Titta 
Date: 



7 October ;£r>/o 



tola'?! to 



Indiana Office of Techno! 



By:_ 



'echnolAgy: 

EX 



Printed Name: Brian Arrowood 
Title: Chief Information Officer 
Pate: IQ-XS— \^ 




Indiana Department oCfttfncation: 

By: 

Dr. Ton/ Bennett 
Superintendent of Public Instruction 



Date: 



10 




State Budget Agi 




APPROVED as to Form and Legality: 
, of the Attfrhev General 





Adam M^Horst/ Director 
Date: //- ST~ / ft 



Gregory?. Zoeller, Attorney General 
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EXHIBIT A 



CTB/McGraw-Hill 
ISTEP+ 

Indiana Statewide Test of Educational Progress Plus 

July 1,2010 
to 

June 30, 2012 
Contract Extension SOW 

Mark Heidorn, Program Director 
Leslie Strickland, Program Manager 
Shelby Gallagher, Program Manager 
Cece Robinson, Program Office Coordinator 



This document contains proprietary information thai belongs exclusively to CTB /McGraw-Hill. 
Disclosure or use in any form is strictly prohibited without written permission. 



Contract No. 05688-KG08-000 



EXHIBIT A 
SCOPE OF WORK 

NOTE: Within this Exhibit A references to "CTB" and "McGraw-Hill, " indicate the "Contractor": 
references to "IDOE " indicate the State and references to "ISTEP+ " indicates the Project 's name under this 
Contract. 
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Table 5 - Printed Materials 14 

Enrollment, Fulfillment, and Delivery 16 

CTB Web-Enabled Services (Navigator™) Deliverables 16 
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Report Generation and Delivery 21 

Table 6 - Report Matrix Spring 201 1 and 2012 21 
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IDOE Reports from CTB 25 

Technical Reports 25 

Error Report 25 

Document Retention 26 

CTB/Indiana Help Desk 26 
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Executive Summary 

This is a two-year extension option to the original Component #3 contract covering the Spring 20 1 1 and 
Spring 2012 administrations. 

Grades 3 through 8 will be included in these test administrations. Both administrations will test E/LA and 
Math for Grades 3-8, Science for Grades 4 and 6, and Social Studies for Grades 5 and 7. 

With the exception of the Student Label and one copy of Student Report, all reports will be online. 

There will be two testing windows as in the previous two years; one for Applied Skills (constructed-response 
items to be administered in early March) and one for Multiple-Choice (late April/early May). 

In Spring 201 1 and 2012, 20% of the student population will take Applied Skills and 20% of the student 
population will take Multiple-Choice online through the OAS application. For Applied Skills, 100% of Math 
and Science will be taken in paper and pencil. 

All content areas of the Multiple-Choice portion of the test will be administered online. Only one online form 
will be available. 

There is a 7-calendar day turnaround time for reports after the receipt of the Multiple-Choice portion of the 
test (late April/early May). If less than 20% of the students test online for Multiple-Choice, the turnaround 
time will be negotiated with the IDOE. 
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Team Meeting Schedule: The Program Management Team Meetings will be held: 

Every Thursday at 10:00- 11:00 a.m. in the Fishbowl CR 

Job numbers: C1007/D1007 Spring 201 1 and Spring 2012 



Table 1- Key IDOE Staff 



Name 


Position 


Ti'leplioni' 


r.maii 


Wes Bruce 


Chief Assessment Officer 


317-232-9054 


wbruceiSjdoe. in.gov 


Michcle Walker 


Director of Student Assessment 


317-232-9051 


mwalketfo)doe.in.epv 


Charity Flores 


Assistant Director of Student 
Assessment 


317-232-9050 


cflores(3)doe.in.eov 


Linda Potter 


Assessment Support Specialist 


317-232-9056 


lDOttert3idoe.in.gov 


Sharon Watts 


Accreditation Assessment & Licensing 
Support Specialist 


317-232-9050 


swats@doe. in. eov 


Michelle Wenzel 


Mathematics Assessment Specialist 


317-234-0503 


mwenzel®doe.in.eov 


Jim Mirabelli 


Mathematics Assessment Specialist 


317-234-5602 


imirabel^doe.in.eov 


Kelly Connelly 


E/LA Assessment Specialist 


317-232-0816 


kconnel lv(33doe. in.eov 


Beth Reel 


E/LA Assessment Specialist 




breelfS>doe.in.eov 


Cindy Hopkins 


Science/Diagnostic Assessment 
Specialist 


317-232-9050 


chopkins@doe. in . eov 


Cheryl Hustedt 


Core 40 End-of-Course Assessment 
Specialist 


317-234-0231 


chustedtfS)doe. in. eov 


Jennifer Thompson 


National Assessment of Educational 
Progress Specialist (akaNAEP 
Specialist) 


317-232-9050 


ithompson^ldoe. in.eov 



Table 2 - Key CTB Staff 



Program Director: Mark Heidom 850-878-3854 
Program Manager: Leslie Strickland x7401 
Program Office Coordinator: Cece Robinson x7193 
Financial Analyst: Sonja Aldisert x7686 
Field Rep: Jerry McCanna 317-581-9404 

Project Managers 
Development: 
Production Lead: 
Manufacturing/Fulfillment: 
Scoring: 

Technology & Research: 

Resource Managers 
Development: 
Production: 
Ancillary: 
Manufacturing: 
Scoring: 

Enrollment, Fulfillment & Delivery: 
Document Processing & Report Generation: 
Software: 
Research: 



Program Manager: Shelby Gallagher x761 1 
PSA: Lisa Staalenburg x6855 

Solutions Manager: Linn Williams x7896 



Krista Bobbin 805-466-292 1 
Leslie Simon x7268 
Will Fredricks 630-789-4584 
Amanda Hill 317-205-0220 
Shiva Doreswamy x7 1 35 



Krista Bobbin 805-466-292 1 
Carolyn Suorsa x6404 
Krista Bobbin 805-466-292 1 
Jennifer Morgan x7895 
Anton i a Deoudes x747 1 
Rich Johnson x7351 

Debbie Mucke x7459 & Shirley Espiritu-Nimis x7260 
Shiva Doreswamy x7 1 35 
Antonia Deoudes x7471 
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Program Description 

Contract duration: 2010-2012 E/LA, Math, Science, and Social Studies 

Fiscal Year Duration July 1-June 30 

Scope of this Design: Spring 201 1 and Spring 2012 Test Administrations 

Document and Materials Development 

Ancillary Materials Development and Production 
Document and Materials Production 
Software Application Customizing 
Assessment Design and Analysis 
Enrollment, Fulfillment and Delivery 
Document and Materials Manufacturing 
Document Processing 
Report Generation and Delivery 
Data Re-verification 
M/WBE Information 



Number of Public Corporations: approximately 293 
Number of Public Schools: approximately 1600 
Number of Non-public Diocese: approximately 11 
Number of Non-public Schools: approximately 270 
Number of Students: approximately S 1 0,000 

Customer Participation 

The IDOE will ensure prompt access to and decisions by the staff of the IDOE, by CTB or subcontractors 
as designated whenever such access and decisions are required in connection with the orderly progress of 
work for mis program. 

The IDOE will participate in review and approval of the following: 

Customer Meetings 

CTB to provide the following: 
Meeting Agendas 

Schedule meetings and arrange for meeting facilities 

Pay meeting facility fees 

Prepare meeting materials 

Lead the meetings with IDOE staff 

Prepare and maintain documentation of procedures used/decisions made 
Provide meeting refreshments and meals 
- . Provide airfare for the IDOE staff 
' - - Provide a per diem for IDOE staff traveling to California 
Provide a rental car for IDOE staff traveling to California 
Provide lodging for IDOE staff in Monterey 

Meetings for this administration: 
Planning Meetings 
Post-Test Meetings 
Status Meetings 
Test Decks 
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Program Management 

Negotiate all contract changes in scope, regardless of cost; document any changes in cost, schedule, or 
required deliverable. Ensure that customer and PM sign the DCF for any changes in schedule or scope . 
Follow up with customer and manage departments to ensure mat the base-lined schedule is achieved, and 
that all requested changes in scope are managed so that the ultimate delivery dates for test and reports are 
met 

Identify risk, rank, and mitigate all process risks that can impact the program schedule or the quality of 
deliverables 

Develop the work schedule and process plans with assigned team members 

Manage the program schedule to ensure that high quality deliverables are provided to the customer on 
time 

Serve as point of contact between the I DOE and CTB departments 

- Submit weekly the I DOE Status Report and Action Item List to the I DOE 

Facilitate weekly team meetings. Provide opportunity for timely communication between departments. 

Keep departments apprised of project status 

Prepare and send correspondence to meeting attendees 

Prepare customer meeting agendas for review by the IDOE 

Schedule and conduct Red Team Reviews \ 

Schedule and conduct with the IDOE the Test Deck Review 

- Arrange IDOE visits to CTB 
Prepare and submit invoices 

- Manage Change Orders and Decision Logs for the contract 
Produce the Quarterly Error Report 

Monitor Minority and Women Owned Business Enterprises contract compliance 

Minority and Women Owned Business Enterprises Contract Compliance 

The Contractor agrees to comply fully with the provisions of 25 IAC 5 and the Subcontractor Commitment 
submitted with the State. No changes may be made to the commitment without the written approval of the 
Minority and Women's Enterprises Division of IDOA. 

The following MBE's and WBE are listed on the Minority and Women's Business Enterprises Division 
directory of certified firms will be participating in this Contract. 

. Table 4 - M/WBE Vendors 



MBE 

or 
WBE? 


Telephone 
No. 


Company Name 


Scope of 
Product/Service 


Utilization Dates 


Amount 


WBE 


317-578-7225 


All Pack Specialists, 
Inc. 


Provide corrugated 
boxes. 


July 20 10-March 2012 


$100,000.00 


WBE 


260-434-0990 


Briljent LLC 


Training services 
including 

technology services, 
on site training and ' 
site survey. 


December 2010- 
April 2012 


$184,320.00 


WBE- 


856-486-0066 


Interactive Printing 
and Packaging 
Corporation (1PAK) 


Printing large print 
books, 

mariipulatives, paper 
bands, stack cards 
and other materials. 


May 2010- January 
2012 


$359,090.00 
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Wot 


j 1 l-o /z-44yu 


Langham Logistics 


1 ransportanon and 
logistics 


septemoer 2\) i u- 
April2012 


1 fAA An/1 

5 1 ,500,000.00 


MBE 


214-550-6124 


Dployit 


Professional Staffing 
ior conunci 
employees 


September 20 10- May 


$2,300,000.00 


MBE 


847-977-0630 


•Creative Printing 
Services, Inc. 


Provide printed 
ancillary test 
materials 


July 2010- June 2012 


$170,000.00 



'New Vendor will replace Solo Printing 

NOTE: The parties hereto understand that the subcontractor amounts noted in the above table have 
been adjusted for the final contract price and represent an increased usage of MBEJWBE entities to 
10% of the contract price. 



The Contractor agrees to submit a copy of the agreement entered into between the Contractor and each 
MBE/WBE subcontractor where the State took the selection of the MBE/WBE by the Contractor into 
- - consideration when issuing the procurement award. The copy of the agreement must be submitted to the 
MWBE Division in IDOA within ninety (90) days of the execution of the contract between the Contractor and 
the State. The Contractor also agrees to send all amendments, changes, and terminations to these agreements 
■ to the MWBE Division in IDOA within ninety (90) days of their execution. Failure to provide a copy of the 
agreement or subsequent amendment, change, and termination may result in exclusion from future State 
procurements. If the Contractor is hot excluded from future procurements, the actions or inactions of the 
Contractor with regard to the above will be taken into account in all phases and scoring in future 
procurements. In addition, the Contractor must obtain the approval of the Division before changing any 
MBE/WBE participation plan submitted in connection with this Contract. 

Program Milestones and Critical Dates 

Spring 201 1 - See Attachment F 
Spring 2012 - See Attachment G 

Out of Scope Work 

The following items are not included in the price or scope of this contract. If desired, they can be added for 
an increased cost. 

Spanish translations of reports 

Secure Barcodes for inventory purposes 

Any report redesign work initiated at the IDOE's request. As it is unclear how many reports will change, 
costs cannot be estimated for the new reports at this time 

- Any production of new items, the piloting of those items, and the associated workshops needed to 
approve the new items are not covered in Spring 201 1 and 2012 and will be charged to the IDOE 
Bulk/Group printing of online reports in INORS. Bulk print of ISR and Image Print is an exception 

- Additional printing, fulfillment, shipping, or scoring of test materials over the amount stated in the Work 
Scope (see Document and Material Manufacturing) 

' - Teachers added to the Range finding meetings 

- Item Alignment Review Meetings 
Specification Meetings 

- Greater than 80% Paper and Pencil Solution for the Multiple-Choice online testing 
Scrambling of items to administer a different virtual form for online Multiple-Choice test (OAS) 

- Practice Tests, Gridded Response Practice Tests, Test Coordinator's Manuals, Teacher Scoring Guides, 
and the Guides to Test Interpretation will not be printed. These will only be developed for online access 
and printing 

- Gl utei- than 20% online students for A p 
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Below is a list of the Decision Change Forms for potential out of scope work: 

PM06-SI 1-12 Secure Barcodes and Inventory 

PM08-SI 1-12 Test Coordinator's Manuals 

PM09-S 11-12 Practice Tests 

PM10-SU SIQ Changes 

PM11-S11-I2 100% Paper and Pencil 

PM12-S1 1-12 OAS Multiple-Choice Virtual Form 2 

PM16-S1 1 Delete INORS Reports) 

PM19-S1 1 De-scope SI 1 OAS Applied Skills OAS Testing 

PM21-S1 1-12 GQE Storage 

PM22-S 11-12 GQE Material Destruction 

PM36-S1 1 Science Specification Meeting 

PM37-S1 1 Science Item Development 

PM38-S1 1 Science Content and Bias Review Meeting 

PM39-S12 Science Pilot Test and Associated Tasks 

PM40-S1 1 Student Barcode Layout Changes 

Development/Publishing Deliverables 

The content developers assigned to the ISTEP+ program will work closely with the IDOE in to build quality 
assessments that meet Indiana Standards and that provide valid, reliable results upon which educational 
decisions can be made. 

(Development Requirements and Test Design Tables are provided in Attachment D) 
Document and Materials Development 

- Prepare Development specifications 

- Develop test design, assessment limits 

- Identify passages and items for Department review 

- Art development and production 

- Produce operational test forms (print form and one online form) 

- Produce manuals and ancillary materials 

- Provide updated PEID tables to Technology and Research 

- Obtain copyright lines 

- Obtain permissions 

- Create custom cover designs 
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Document and Materials Production 

Receipt of manuscript by Art and Production 

Creation of first pages 

Creation of second pages 

Customer review and approval 

Materials Integration Review (MIR) 

MIR administration 

Review and documentation of results 

Creation of third pages and quality check 

Query resolution 

QA corrections made and reviewed 

Pages submitted for prepress 

Flight Check, postscript files, and convert to PDF 

Review of prepress output 

Release to Manufacturing 

PEID files and item objective maps to Research and Technology 
Scoring guides and anchor lists from editors to Research and Handscoring 

Assessment Design and Analysis Deliverables - Research Tasks 

Develop Operational school corporation list 

- QA of PEID files 

Create, QA and deliver pre-equated scoring tables for OP and Braille to Technology (Final PAR Files: 
Operational and Braille to Statistical Services for scoring tables creation all grade/content) 
Develop and deliver IPI cut scores (OP and Braille) to Technology ( 1 table per grade/content area = 1 8 
OP; Braille count unknown) 
Develop Analyses Specifications 

Receive data for analyses in two waves: Applied Skills (Phase 1 ) and Applied Skills plus Multiple- 
Choice (Phase 2) 

Use of standard Research Process Upgrade for: Classical Item Analyses, DIF, Calibration, and Equating 
for all operational items 

Perform post-equating analysis with production slice data and prepare a report to I DOE. Worst case 
scenario, if the pre-equatihg does not hold good upon analysis of the data, create and deliver updated item 
parameters/tables (post equating analysis will NOT include OAS test taking student population) 

- QA4IRT - receive GRT from Scoring, create DAT files, run QA4IRT 
Red Team review analyses of GRT only 

- Complete 1 Technical Report that covers analyses for Operational data and deliver to customer via secure 
FTP, and deliver 3 hard copies and 2 CDs to Programs to deliver to I DOE and field representative 

- Create, QA, and Deliver ItemWin DAT (or similar) files to Publishing 

- Form Selection training, specifications, and reviews 
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Test Design — General Information 

- For Spring 201 1 and Spring 2012 there are 2 test administrations; Applied Skills and Multiple-Choice, 
one form each. While the Applied Skills books and the grade 3 Multiple-Choice books are scannable, the 
grade 4-8 Multiple-Choice books will be non-scannable with scannable answer documents 

All grades will be testing E/LA and Math; Grades 4 and 6 will also be testing Science; and Grades S and 7 
will also be testing Social Studies 

Grade 3: E/LA, Math 

Grade 4: E/LA, Math, Science 

Grade 5: E/LA, Social Studies, Math 

Grade 6: E/LA, Math, Science 

Grade 7: E/LA, Social Studies, Math 

Grade 8: E/LA, Math 

Note: Test will be administered in order; Math then E/LA then Social Studies (or Science). 

- In Spring 201 1 and 2012, 20% of the student population will take Applied Skills and 20% of the student 
population will take Multiple-Choice online through the OAS application. For Applied Skills, 100% of 
Math and Science will be taken in paper and pencil as they are not available online. All content areas of 
the Multiple-Choice portion of the test will be administered online. Only one online form will be 
available. There is an online Multiple-Choice Practice Test available in grades 3-8 

- The Math Reference Sheet will be perforated in the Applied Skills and Multiple-Choice operational books 
for Grades 6-8 and a separate sheet for Large Print books for Grades 6-8 

The Practice Tests for Grades 3-4 is 2 separate tests as follows: an Applied Skills Practice Test and a 
Multiple-Choice Practice Test which will be available online. The Practice Tests will not be printed by 
CTB 

There will be a Gridded Response Practice Test for Grades 6 through 8 which will be available online. 
The Gridded Response Practice Test will not be printed by CTB 

- Gridded Response items for operational tests for Grades 6 through 8 
Large Print for the operational tests at Grades 3-8 

Braille for the operational test at Grades 3-8 and ancillary materials to include Practice Tests, Reference 
Sheets, and Manipulatives 

- CDs are available for Braille, Large Print, and IEP students (Large Print and IEP CDs are the same) in 
Grades 3, 4, and S. The Braille and Large Print CDs will ship with the materials. 

- The Large Print and Braille materials and IEP CDs will be ordered via the <www.ctb.com/istep> 
webpage 

Copyrights/Ownership 

All work for ISTEP+ is custom 
The IDOE owns all items 
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Manufacturing Deliverables 

- Operational test books (commodity codes, forms, and quantity) 
Test book colors agreed upon between the IDOE and CTB 
Examiners Manuals in grades 3-8 

Large Print Materials 
Braille Materials 

- Audio CDs grades 3, 4, and 5 
Group Information Sheets 
School Group Lists 

- Test Coordinator's Kits (excluding Test Coordinator's Manuals which are only available online) 
Enrollments 

Samples 

Stack cover cards 

Bands 

Poster 

Document and Materials Manufacturing 

- Test materials are shrink-wrapped and packaged by 6's and by 12*s. All initial shipments will be sent in 
packages of 6 and 12. CTB will work to exhaust all packages of 12 in initial shipments. TMA shipments 
will be filled with packages of 6 and 12 (until 12's are exhausted) 

Cover Memos and ancillaries will be updated by Program Management and approved by IDOE prior to 
distribution 
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Table 5 - Printed Materials 



Materials 


Sarins 201 1& 2012/AS 


SDrine 201 1 & 2012/MC 


Grade 3 Test Books Printed (E/LA and Math) 


83K 


83K 


J** .'_ * _ a . f"» _ - 1_ _ w^_» __ . ■ frit A 1 *_»L *"* ^ * ^ V 

Grade 4 Test Books Printed (E/LA, Math, Science) 


80K 


80K 


Grade 5 Test books Printed (h/LA, Math, bo) 


77K 


77K 


Grade 6 Test Books Printed (E/LA, Math, Science) 


77K 


77K 


Grade 7 Test Books Printed (E/LA, Math, SS) 


77K 


77K 


Grade 8 Test Books Printed (E/LA and Math) 


81K 


8IK 




c nr ino Mil JL 7ni?/AC 








20% Online 


Grade 3 Test Books 20% Online Printed 
(Math tested p/p in book but also includes E/LA) 


20K 





Grade 4Test Books 20% Online Printed 

(Math and Science tested p/p in book but also includes E/LA) 


20K 





Grade S Test Books 20% Online Printed 

(Math tested p/p in book but also includes E/LA and SS) 


19K 


o 


Grade 6 Test Books 20% Online Printed 

(Math and Science tested p/p in book but also includes E/LA) 


I9K 





f^nJ. 1 Tap* DaaL* tfWJL /"^*li>u Pi I_ L rj-1 

made / i est books zxrfy uniinc ranted 

(Math tested p/p hi book but also includes E/LA and SS) 


19K 





Grade 8 Test Books 20% Online Printed 

(Math tested p/p in book but also includes E/LA) 


20K 







SDrine 201 1& 2012/AS 


SDrine 201 14 2012/MC 






20% Online 


Grade 3 Answer books Printed 








Grade 4 Answer Books Printed 





' 80K 


Grade 5 Answer Books Printed 





77K 


Grade 6 Answer Books Printed 





77K 


Grade 7 Answer Books Printed 





77K 


Grade 8 Answer Books Printed 


- 


81K 






Sarins 201 1& 2012/AS 


Sorine 2011& 2012/MC 


Grade 3 ft of Operational Forms 


1 


1 


Grade 4 ft of Operational Forms 


1 ■ 


1 


Grade S ft of Operational Forms 


1 


1 


Grade 6 ft of Operational Forms 


1 


1 


Grade 7 ft of Operational Forms 


1 


1 


Grade 8 ft of Operational Forms 


1 


1 


GTI 


Sorine 201I& 2012/AS 


Sorine 201 1& 2012/MC 








All Grades 


Y (PDF online) 


Examiner's Manual 


SDrine 201 1& 2012/AS 


Sorine 2011& 2012/MC 








All Grades (E/LA & Math Gr 3-8, Sc Gr 4,6, and SS 5,7; 1 
book per grade) 


144K (paper 24K) 


144K (paper 24K.) 


Teacher's Scoring Guides 


Sorine 201 1& 2012/AS 


Sort.* 2011* 2012/MC 








All Grades (E/LA & Math Gr 3-8, Sc Gr 4,6, and SS 5,7) 


Y (PDF online) 




BraiUe 


Sorine 201 1& 2012/AS 


Sorine 201 1& 2012/MC 


All Grades Test Books 


Y 


Y 


Grade 3 Practice Test 


Y 


Y 


Gridded Response Practice Test Grades 6-8 


Y 


Y 
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Rulcr/Manip Grades 3-4 


Y 


Y 


Ruler/Manip Grades 5-8 


Y 


Y 


Protractor Manip Grades 6-8 


Y 


Y 


Reference Sheet 6-8 (S 1 1 , S 12) 


Y 


Y 


CDs Grades 3-5 


Y 


Y 








Large Print 


Serine 201 1& 2012/AS 


Sonne 20II&2012/MC 


All Grades Test Books 


Y 


Y 


Grade 3 Practice Test 


Y 


Y 


Gridded Response Practice Test Grades 6-8 (SI 1, SI2) 


Y 


Y 


Rulcr/Manip Grades 3-4 


Y 


Y 


Rukr/Manip Grades 5-8 


Y 


Y 


Protractor Manip Grades 6-8 


Y 


Y 


Reference Sheet 6-8 


Y 


Y 


CDs Grades 3-5 . ~ " 


Y 


Y 


AnclHaries 


Snrine 201 1& 2012/AS 


Snrine201I&2012/MC 








Grade 3-4 Practice Test 


Y (PDF Online) 


Y (PDF Online) 


Grade 3-4 Manipulative 


Y 


Y 


Grade 5 Manipulative 


Y 


Y 


Grade 6-8 Manipulative 


Y 


Y 


Gridded Response Practice Test Grades 6-8 


Y (PDF Online) 


Y (PDF Online) 


Reference Sheet Grades 6-8 


Y 


Y 


Test Coordinator's Manual 


Y (PDF online) 


Poster 


Y (paper 3.5K) 


Y (paper 3.5K) 


Paper Bands 


Y 


Y 


Stack Cover 


Y 


Y 


SGL 


Y 


Y 


GIS 


Y 


Y 


Return Shipping Labels 


Y 


Y 


Cartons 


Y 


Y 


•total print quantities include both AS sections in a 


x>ve table 



♦there is only one answer book form available 
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Enrollment Fulfillment, and Delivery 

Prepare Fulfillment/Traffic Specifications 
Assist with the Enrollment process 

Fulfillment will receive an enrollment file from the I DOE with student enrollments by corporation, 
school, and grade 

- The Enrollment File will need validation from the I DOE regarding updates and discrepancies 

In addition to the Enrollment File the IDOE needs to provide a list of reconfigured, closed, newly 
opened, and renamed schools 

- Any changes to the Enrollment File, after the final Variance Review, by the IDOE could cause delay in 
material deliveries. These late deliveries are not subject to liquidated damages. 

Materials packaged by corporation, school, non-public school, and charter school 

- Order Forms for enrollment collection for Large Print, Braille, and audio CD's can be found on 
<www.ctb.com/istep > webpage for online ordering through the help desk 

Enrollment contacts at IDOE are Michele Walker and Charity Flores 
Packing Lists are to be used with all shipments 

CTB Web-Enabled Services (Navigator™) Deliverables 

For the Spring 201 1 and Spring 2012 administrations the following Navigator™ Services will be provided: 
Online Short/Adds (TMA) 

Entry will be permitted to test coordinators only 

Three administration windows will be created, i.e. school corporations, non-public schools, and charter 
schools 

Corporations, charter schools, and non-public schools will continue to use the username/password 

provided from previous applications or new userid/password as distributed through DEx implementation 

schedule, whichever holds pertinent prior to the TMA window opening 

Help desk can issue a new username/password 

Cover memos and user guides will be provided by CTB 

The customer will determine the messages posted in the message box 

TMA must include the functionality of ordering Braille and Large Print materials for non-public and 
charter schools 

Post-Test Modifications (Record Editing System) 

RES will include a user-friendly data file layout 

RES window will be between the two testing windows and is only applicable for operational data 

received from Applied Skills test window. 

Cover memos and user guides will be provided by CTB 

Shipping Service 

Shipping Mode: Freight Carrier or UPS Ground service based on shipment size for efficiency (unless 
contract delivery in-site date cannot be met) for the material deliveries to corporations, non-public schools, 
and charter schools 

TACS used: No online scheduling system will be used 
Return Label Color coding: 

Phase 1 : Yellow (Applied Skills - scorable), Orange (Applied Skills Validation {new name TBD} 
Schools - scorable), and Blue (Unused/Inventory) 

Phase 2: Purple (M-C Scorable), Tan (M-C Validation {new name TBD} Schools - scorable), and 
Blue (Unused/ Inventory) 
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Any early shipments? Yes, Validation (new name TBD) corporations will be picked up early as follows: 
There will be two windows of Validation (new name TBD) pick-ups; one for Applied Skills 
materials in March (window 1) and one for Multiple-Choice materials in May (window 2) 

Preference for carrier selection for pick-up of materials? 
Langham Logistics 

Online Assessment System (OAS) 

- Perjh^f^^ 

Error whether less than on 

Stu^entpricc' for the': discre^c^^H ib^detlrmin^ 

If less than 20% of the student population participates online a minimum 1 80 day notification is required 
from the IDOE to CTB in order to provide printed test materials to the students who are no longer testing 
online. The costs for the additional printed materials will be charged to the IDOE 
The 20% participation requirement is entire grades at each school except for Large Print and Braille 
See Attachment C for detailed OAS scope information 

All product functionalities/tools supported in S10 will continue to be supported 

Bom Applied Skills and Multiple-Choice operational tests are in scope for OAS for Spring 201 1 and 2012 
Comparability Study 

A comparability study was conducted utilizing Spring 2009 results. The results of this study serve as the 
basis for OAS administration for 2010 and 201 1 administrations 
Requests for additional studies will be out of scope 

Software Application Customization 

Prepare Technology Specifications, scoring and reporting business rules, and mockups, including data file 
layouts. Report PDFs are no longer provided. Only data files to both IDOE and CTB online reporting 
systems 

Develop image program for scanning new test forms and handoff image PDF files to online reporting 
systems 

Develop Edit/Raw Score programs for test forms 

- Develop Handscoring programs for scoring the Applied Skills tests 

- Develop Scoring and Reporting programs for new test forms, including functionality related to the match- 
merge for Applied Skills and Multiple-Choice tests 

Develop customized tools to assist Scoring with data clean-up tasks (NCNT by book, Delete by OpUnit 
utilities) 

- Develop SMF extract functionalities to OAS and the INORS and Parent Network systems 
Load and QA final scoring tables and hems contributing to final operational scale score 
Prepare for and attend Red Team Review 

Prepare for and lead Test Deck review with IDOE 

Work with the Indiana Online Reporting System (INORS) and Parent Network components and provide 
all data handoffs necessary to enable those systems to report ISTEP+ data 
Develop and setup EHS rescore functionality and custom EHS reports 
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Scoring Deliverables 

Prework Requirements 

- Prework required? 
Precode Group Information Sheet (GIS) 

- School/Group List (SGL) 
Return Box Labels 

Student Barcode Labels and Student Rosters 
Precode Barcode 

Specific information regarding barcodes will be provided in the Technology Specifications 
Barcodes will be provided for both the 201 1 and the 2012 test administrations in 3 phases. Edit software 
will be used. The file layout also includes the Teacher Name, Teacher ID number, and Social Studies 
Accommodation fields. The barcodes are sorted using the "file in -file out" method. One copy of the 
roster is sent to the corporation and also includes the Teacher Name. CTB will use the label design that 
has been previously approved by the IDOE 

- A sample barcode file will be received from the IDOE to Scoring approximately 1 days before the 
scheduled delivery date of the Phase 1 barcode file for a QA check to make sure the format of the data is 
correct 

Barcode Phases: 

Phase 1 to be sent before Applied Skills testing (2 Student Barcode Labels included; 1 for the Applied 
Skills Assessment and the additional as a backup in case first barcode label is damaged. 
Phase 2 to be sent the week of testing for Applied Skills of only those students who weren't included in 
Phase 1. 

- . Phase 3 will include a set of barcode labels for all students provided in Phases 1 and 2 along with all new 
- students provided in the Phase 3 barcode file from IDOE. This shipment will be sent to the corporation 

prior to the MC testing and will include 2 labels for each student 

Ancillary Systems 

There are multiple PDBs used (public school corporations, non-public schools, and charter schools) for 
the ISTEP+ program. The PDBs will contain the following information: 

Corporation number and school number 

Flag indicating that all schools are public (any private schools) 

Flag indicating participating schools 

Corporation name and address 

Enrollment per grade by site 

School name and address 

- Contact person name at the school 
Phone number of contact person 



^ Yes DNo 
El Yes QMo 
^Yes DNo 
^Yes DNo 
E]Yes DNo 
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Document Processing 

Document Processing: Operational for 2011 -2012 

In Spring 2010 applied Skills field test items were piloted to develop Operational items for Spring 201 1 
and Spring 2012 testing. 

Scoring will prepare detailed specifications: ISTEP+ Project Scope document, PDB Specifications, Pre- 
processing Specifications and Print Profiles specs will be developed production processing of Scoring 
Kit materials (SGL, GIS and Return labels) and Student Barcode labels. Document Processing 
Specifications for Applied Skills processing and a separate set of specs for Multiple Choice will be 
developed. 

Prepare Prework for document processing (Scoring Kits and separate Student Barcode label shipment) 
Review PDB file from Fulfillment and compare to previous year to ensure accuracy and will report 
discrepancies to Program Management for variance verification 

- Set up Organization testing programs and build hierarchy for processing 

Deliver Test Coordinator Kits for both windows of testing: Scoring Kit items will be submitted for print 
and will be added with other TCK items at the large scale Pick and Pack. Scoring kits(SGL/GIS and 
Return labels) for both Applied Skills and Multiple Choice will be included in the same shipment 
Deliver 3 phases of Student Barcode labels. First set of Student Barcode labels to' be delivered before 
the Applied Skills test window. Second phase will only consist of those additional students who weren't 
included in the first barcode file. Third phase will include a complete set of all students provided in 
Phases 1, 2, and 3. The third phase is to be delivered before Multiple Choice test window 
Provide Technology with Student Master File (phase 1 and 2 student barcode files) for deployment for 
window 1 Applied Skills Scanning. Also provide final consolidated Student Master file of all Phases of 
student Barcode files received from I DOE (1-3). 

Provide OAS group with extract files for corps/schools testing on-line, (used to set up hierarchy for 
Multiple-Choice online testing). 

Provide the INORS and Parent Network teams with Pre-ID extract files to support the creation of Parent 
Network accounts 

Scoring will lock down Mainframe Prework: Tracking records , school group list application, stack 
headers and prework print on demand for document processing prior to the Applied Skills receiving 
window 

- Process Scoring test deck data through Operations and Handscoring for user acceptance testing with to 
ensure front-end and backend software applications are functioning as intended. 

Process IDOE test deck through front-end, after final Production software is released: receive. 

- Operational Data: Unique op units will be assigned for the processing of the Validation (new name 
TBD) sample data (Operational) as well as all data coming from OAS. (to ensure that data can be 
identified after being exported). 

- Validation (new name TBD) data will be prioritized for Research prior to regular operational data. 
There will be two cycles of Validation (new name TBD) data, one in the Applied Skills window and the 
other in the Multiple-Choice window. All Operational data will be processed after Validation (new 
name TBD) data completes. 

- Upon retrieval of Applied Skills material, Scoring will receive, log in, image scan and edit data through 
WinScore and hand score items through EHS system for Applied Skills during Window 1 . 

- After export, data will bridge to me mainframe. 

Scoring will verify completeness using the Monarch system. 

- After A/S data completes through the Handscoring system, scored CR items will be exported from EHS 
and Op units FTP'd to the mainframe. 

- Scores loaded to EISS. 

- Provide ISO with Extract file for RES and Post res Files as required 

- -Upon retrieval of Multiple-Choice, Answer documents will be receive, logged in, scanned, scored and 

data edited through WinScore and exported to the Mainframe during Window 2. 

Exhibit A, Page 19 of 66 



Contract No. 05688-KG08-000 



The match merges process will be triggered by the bridging of multiple choice data. 
Load Res File to EISS after res window closes 

Image Print: Development will work with Technology to pre- identify image print page specifications 
Scoring will verify corporation completeness for image print roll-up and will report problematic issue to 
Technology for clean-up. After each corporation completes through Scanning and all their op units export 
from WinScore. Student images go through book roll up process. A validation check is performed against 
the number of books exported from WinScore as compared to the number of books in the book roll-up 
summary report Images automatically FTP to holding site using a paperless image print solution for the 
INORS and Parent Network systems. 

After Applied Skills and Multiple-Choice clean-up/match merge, Scoring will deliver Matched 
Validatio n (new name TBD) data to Research in GRT format using final tables. 

201 1 and 2012 are pre-equated with a post equated verification checks. Research will use the Validation 
(new name TBD) sample to do a validity check in 201 1 and 2012. 

After all Operational scoring and match merge completes, a final GRF and reports for Red Team Review 
will be provided to Research 
Participate in Red Team Review. . 
After Red Team is approved by IDOE and final sign-off 
Reports are produced and released in 4 phases, 

Phase 1 reports will be released through INORS and the Parent Network utilizing the following files for - 
INORS and Parent Network: ( GRTs, SUMTs, PPR and CPRs files) to support the IN Online Reporting 
System (PPR, CPR, and Rosters) and to support the Parent Network (Student reports and Image Print) 
Image Print (INORS and the Parent Network) 

Phase 2 Label and paper Student Report. Produce and ship paper reports. Reports (labels) packaged by 
school and shipped to schools or corporations. Unscannable image prints will be printed, picked, packed 
and shipped separately by the Document Processing group. 
Phase 3 Summary reports on line 

Phase 4 release of Post Rescore reports through INORS and Parent Network and delivery of final GRT 
after Post Rescore to IDOE and new Disaggregation posted 

Handscoring 

For the Operational Grades 3-8, the 5% sample for 2nd reads is for research purposes and will be read to 
resolution: 

- Condition Codes will be scored by evaluators with approval from an expert evaluator: 
A = Blank/no response 

B = Illegible/unable to score 

C = Written predominantly in language other than English/unable to score 
D = Insufficient response/unable to score/copies from text 
E = Response not related to test question or scoring rule 
NOTE: All condition codes convert to points obtained 

- Condition Codes 

Multiple part items cannot receive a condition code and a score with the exception of condition code E 
(off topic/Response not related to test question or scoring rule for E/LA, Writing, Social Studies and 
Science only). Social Studies condition codes will be the same as ELA and science condition codes, 
i.e., A-E. For ELA, a student can receive condition code E for WA and a score for LC. 
Applicable condition codes for Math will be A-D. All math items will receive 2 scores. Math items 
can't receive different condition codes for each part of the same item. 

- If an evaluator is released for failing to maintain standards, the IDOE and CTB will discuss whether 
his/her reads for a particular RIB will be reset back to the time at which they fell below standards or reset 
back to me beginning. 

- If a student completely erased his/her response, a condition code A will be assigned. If there is an 
incomplete erasure, the evaluator will assign a score based upon the non-erased section. 

Image Printing Systems are used 

- 1 item RIBS 
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All CR hems will be scored using a holistic rubric except for Science and Social Studies. 
All scoring will be done at the CTB site in Indianapolis 

Rescore Process Spring 2011 and 2012 ' 

- The INORS team will load students into the INORS system. 
Images will be available in the INORS system. 

Undetermined Student Reports will be on line. Teacher will determine if students require rescoring. 
Rescore requests will be made on line. 

The INORS team will provide the rescore file from the INORS system which will be uploaded to EHS. 
Experienced Handscoring E valuators will review training papers and take a qualification round. After 
successfully qualifying, e valuators will perform a blind read of the items to be rescored. After completing 
the blind read, any scoring disagreements between the students' reported score and the blind rescore will be 
identified and submitted to the content area Supervisors) for review 

The content area Supervisors) will conduct an in-depth analysis of all the items in which there is a scoring 
disagreement After reviewing and analyzing the student response(s), the content area Supervisors) will 
assign final scores to all contested items. If this score differs from the student's reported score, the 
Supervisors) will enter a new score and document the reason for the error and assign one of the eight (8) 
codes for rescore. 

Rescore data will be exported from EHS and scores loaded and QA'd 

Scores will be loaded and data QAM. Report of Rescoring will be Red Teamed and approved for print. 
Scoring will PDF the Report of Rescoring and forward reports to RR Donnelley for printing. 
The print vendor will print reports for all students whose scores have been changed during the rescore 
period. Only students with score changes will receive new Student Reports and Student labels. 
Post rescore files will be delivered to the INORS and Parent Network teams. 

- Updated e-files to 1DOE 

Report Generation and Delivery 

Table 6 - Report Matrix Spring 2011 and 2012 



Grade 


Report Name 


Description 


Mode 


Paper 
Copy 


INORS 
Online 


INORS 
Print on 
Demand 
(PDFs) 


Parent 
Network 


3-8 


Student Report (STUR) Side 1 


Overall scale scores, 
passing levels, IP1 
scores by objective, 
information about 
areas of student 
strength and 
weakness 


Class 


1 




X 


X 


3-8 


Student Report (STUR) Side 2 


Applied Skills 
information, points 
achieved by item, 
points possible, 
condition codes 


Class 


1 




X 


X 


3-8 


ISTEP+ Label (LABL) 


Scale scores, 
performance levels 


Class 


1 








3-8 


Class Academic Standards 
Report (CPR) 


Lists students* 
mastery/non- 
mastery of standards 


Class 




X 






3-8 


Academic Standards Summary 
(ESSM) 


Compares 1 school 
to corporation - 
points possible, IPI 
at pass, mean no. 
correct, mean IPI, 
difference number 


State 
Corp 




X 
X 
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Grade 


Report Name 

r 


Description 


Mode 


Paper 
Copy 


INORS 
Online 


INORS 
Print on 
Demand 
(PDFs) 


Parent 
Network 






mastery, percent 
mastery 












3-8 


Group Academic Standards 
Summary (GESS) 


Same as Academic 
Standards Summary 
- compares multiple 
schools to 
corporation 


Stale 
Corp 




X 
X 






3-8 


Proficiency Roster (PPR) 


Lists students and 
scale scores, 
performance levels, 
HOSS, LOSS, 
noTPCT at each 
performance level 


School 
Class 




X 
X 






3-8 


Undetermined Status Roster 
(UDTR) 


Explains students' 
undetermined status 


School 




X 






3-8 


Disaggregation Summary Report 
(DISA) 


NoTPCT of students 
in each performance 
level by 

disaggregated group 


State 
Corp 
School 




X 

X 
X 






3-6 


Class Proficiency Grouping 
Report (CPGR) 


Lists students in 
matrix by 
performance level 
achieved for E/l.A, 
Math, Science, and 
Social Studies 


Class 




X 






3-8 


Applied Skills Frequency 
Distribution (ASFD) 
Page J 


Distribution of 
students by points 
obtained on Applied 
Skills items 


State 
Corp 
School 




X 

X 
X 






3-8 


Applied Skills Frequency 
Distribution (ASFD) 
Page 2 


Distribution of 
students by 
condition codes on 
Applied Skills items 


State 
Corp 
School 




X 

X 
X 






3-8 


Academic Standards Frequency 
Distribution (STFD) 


Distribution of 
students by scale 
score for E/LA, 
Math, Science, and 
Social Studies 


State 
Corp 




X 
X 






3-8 


Proficiency Performance 
Summary (SPPR) 


Lists students in 
matrix by 
performance level 
achieved for E/LA, 
Math (All), Science 
(Grades 4 and 6) 
and Social Studies 
(Grades 5 and 7). 
Summary data 
includes HOSS, 
LOSS, mean scale 
score, and SD 


Stale 
Corp 
School 




X 

X 
X 






3-8 


Rescore Request Form (RRF) 


Lists students that 
did not pass by 
content area and the 
items for which they 
did not receive full 
credit 


School 




X 







Exhibit A, Page 22 of 66 



Contract No. 05688-KG08-O00 



Grade 


Report Name 


Description 


Mode 


Paper 
Copy 


INORS 
Online 


INORS 
Print on 
Demand 
(PDFs) 


Parent 
Network 


3-8 


GRT 


Electronic .csv 
Format Download 
(Note: Both Public 
Corporations and 
Non-Public Schools 
will be posted) 


Corp/ 
Diocese 
Non- 
public 
School 




X 
X 






3-8 


GRT File Layout 








X 






3-8 


Media Report (MEDR) 


Lists percentage of 
students in a matrix 
by performance 
level achieved for 
E/LA and Math. 
Compares school to 
state by grade at 
Corp and School 


State 
♦(E- 
file) 




- 






3-8 


Disaggregation Summary Report 
(DISA) 


NoiPCT of students 
in each performance 
level by 

disaggregated group 


State 

*(E- 

file) 

Corp 

*(E- 

file) 

School 

(E-file) 










3-8 


Summary Tape 
(SUMT) 


Electronic Summary 
File 


State 

*(E- 

filc) 

Corp 

*(E- 

file) 

School 

•(E- 

file) 










3-8 


GRT 
(GRT) 


Electronic Fixed 
width Text Format 
Download run by 
class mode 


State 
*(E- 
file) 










3-8 


Perfect Scores Report 
(PFSR) 


Students are those 
students who have 
scored perfect 
scores in the 
respective content 
area (this report is 
used at Test Deck) 


State 


1 
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Delivery of Score Reports 

CTB will deliver Phase 1 reports through INORS and Parent Network within seven days of receipt of the final 
Multiple-Choice answer documents from the Indiana corporations to the Indianapolis Scoring Site. The 
seven-day reporting schedule is dependent on pre-equating of the ISTEP+ test forms. Prior to release of 
results, CTB will present IDOE with a review of pre-equated results in comparison to the results of a post- 
equating analysis prior to scheduled release of scores. Turnaround time for reporting is also dependent on the 
state of documents and data received from corporations and schools and data clean-up protocols agreed-upon 
by the IDOE and CTB. Factors that will impact the seven-day turnaround time include: 

- Less than 20% of students taking ISTEP+ Multiple-Choice online through OAS 

- A decision by the IDOE to use post-equated results 

Unanticipated test materials or data, outside of the control of CTB, that would impact processing 

The following are some factors that may impact the seven-day turnaround for individuals or groups of 
students. Program Management will contact the IDOE within 12 hours if any documents received fail to meet 
CTB's basic processing standards and discuss a process of resolution agreeable to both parties: 

Barcode to Biographic discrepancies - Written name does not match affixed barcode label or incorrect 

label affixed to the student document 

Used wrong pencil type/ student writing to light - Multiple-Choice responses would not be picked up 
Students tested in ink - creates undetermined situations which generate customer queries 
Procedures not followed for handling, packaging, and return of non-standard documents - Large Print, 
computer generated, and Braille - creates undetermined situations which generate customer queries: 

Non-standard documents not listed on the SGL in the comment section. 

NSR is not indicated in response section of books 

Pastel color not used for computer generated responses - removes option of easy detection 
Braille students returned with regular population instead of separately or partial material sent back as 
Braille - on search mission to locate documents which introduces duplicate records 
Late return of Braille - reverse translation can be affected 

Note: For issues not discovered they will be addressed during customer query phase (during the rescore 
window) 

Unreported issues or problems not flagged that are related to processing: 
Students tested in 2 books 
Students tested in each others book 
Ripped, torn, or missing pages 

Wet, damaged, or contaminated materials not properly noted or packaged 
Manipulative, staples, loose pages, or scratch paper left in booklets 
Improper packaging of material: 

SGL not in box 1 of each school. 

Stack cover cards not numbered correctly. 

Placing the GIS under the wrong stack cover card. 

Boxes not numbered properly by school by corp. 

Scorable and non-scorable materials in wrong box 

Scorable and non-scorable mixed together in the same box 

Using custom boxes to return shelf or ILEPA material. 
Materials sent for additional (unexpected) schools which were not in the database 
Late return of test materials 
Placing the wrong color return label on the box: 

Scorable materials packaged in blue label boxes 

Unused materials packaged with scorable materials 

Not properly labeling "Do. Not Score" material 

Placing "Do Not Score" material with live scorable materials 
Incomplete shipments of returned test materials 
Missing expected school(s) 
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Photocopied scannable documents: 
Test booklets 

- Group Information Sheets 

SGL - GIS mismatches: SGL does not match the GIS 1 00%/ SGL data does not match scanned data: 

- Teacher name, grade , and student count 

Altered school name, sch number, and corp. number 
Case count discrepancies - SGL verses GIS does not match 
No SGL/ no GIS received for processing 

- Additional groups received but not listed on the SGL/ teachers listed but not received/ multiple groups 
listed but only 1 group received/ one group listed and multiple groups received 

Student's switching from online paper and pencil and vice-versa resulting in duplicate records 

In the event that these or other factors affect the reporting turnaround time, CTB will negotiate in good faith 
with the IDOE to deliver reports as soon as possible given conditions for each administration. 

IDOE Reports from CTB 

Technical Reports 

- The Operational Technical Reports will be provided to IDOE approximately 3 months after report 
delivery for each administration. Specific dates are yet to be determined. 

Error Report 

- On a monthly basis, Program Management will collect data from all regions about the number and type of 
errors that have occurred on custom programs in the following categories. This information will be 
presented to the IDOE on a quarterly basis verbally without identifying the name of the custom programs 
involved. If the IDOE has any specific concerns that are not addressed by the explanation provided by 
CTB, CTB will request permission from the customer in question to provide contact information to 
IDOE. The IDOE will sign a statement acknowledging the receipt of each verbal report. 

Test Content: A Test Content Error is defined as the inclusion of incorrect or inappropriate item(s) in ah 
assessment such that the validity or reliability of the assessment was affected, and correction of which - 
caused significant delay of the program and/or resulted in public comment. 

- - Test Materials Production: A Test Materials Production Error is defined as an error in printing such as 

incorrect pages or missing pages that affected the validity of the assessment or correction of which 
caused significant delay in production such that the testing was jeopardized and/or resulted in public 
comment. 

- Test Materials Delivery: A Test Materials Delivery Error is defined as an error in packaging, spiral in g 
of booklets, or an error in the delivery of booklets that significantly affected the validity of the 
assessment, or delayed the testing dates and/or resulted in public comment. 

Incorrect Information: Incorrect Information is defined as information that is printed in an assessment 
report that includes incorrect scores on student documents, incorrect school, corporation and/or state data 
and/or resulted in public comment. The cause of these incorrect scores must be found to be a result of 
errors internal to CTB and not as a result of school, corporation or state actions or failure to act. 
Delay in Report Delivery: Delay in Report Delivery is defined as unapproved late delivery of reports 
that significantly affect a school, corporation, or the state's ability to meet internal and external deadlines 
and/or resulted in public comment. 
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Document Retention 

Used test materials will be retained as noted in the table below until approval given by IDOE 

Unused test materials will be retained as noted in the table below until approval given by IDOE 

DO NOT SCORE material should be separated from inventory material and retained with Operational and 

used data until approval given by IDOE 

Used materials will be retrieved upon request 

Written authorization from the IDOE is required prior to destruction 

Approval from the IDOE is required for all document destruction based on the following schedule: 



Grade 


Material 


Retention Time 


3,4,5,6,7,8 


Operational — used + 
Do Not Score documents 


One year 


3,4,5,6,7,8 


Non-consumable and unused 
materials 


6 months 


3,4,5,6,7,8 


Unfulfilled materials (left at 
vendor) 


30 after test window 
close 



CTB/Indiana Help Desk 

- CTB/Indiana Help Desk: 

- Times available: M-F 7:30 a.m. to 5:00 p.m. EST 
Accepts and processes calls for short/adds 

Accepts and processes calls for Large Print and Braille materials and updates spreadsheet 

Accepts and processes calls for scoring issues, reports issues,- and material shipments 

Make customer calls for information needed 

Provides updates for the weekly ISTEP+ team status report 

Creates WebEx training recordings 

Attachments 



For additional information please reference the attachments listed: 

- Attachment A - INORS Work Plan Spring 20 1 1 and 20 1 2 

- Attachment B - The Parent Network Work Plan Spring 20 1 1 and 20 1 2 

- Attachment C - OAS Work Plan Spring 201 1 and 2012 

- Attachment D - Single Sign-on Spring 201 1 and 2012 
Attachment E- Item Development Spring 20 11 and 2012 
Attachment F - Program Milestones and Critical Dates Spring 201 1 
Attachment G - Program Milestones and Critical Dates Spring 2012 
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Attachment A 

Indiana Department of Education Online Reporting System (INORS) 
Spring 2011 and Spring 2012 Work Plan 

Overview 

In the Spring of 201 1 and the Spring of 2012, the ISTEP+ Assessment will be administered and reporting will 
be delivered through the INORS. The INORS solution will be released in iterative stages throughout the 
Spring of 201 1 and Spring 2012 and will include reporting for the ISTEP+ Assessment in Grades 3-8. 
Existing Spring 2010 ISTEP+ reports will remain available during the Spring 201 1 deployment. Existing 
Spring 201 1 ISTEP+ reports will remain available for the Spring 2012 deployment 

The following tables provide a high level description of the features included in INORS: 



• Online Reporting for ISTEP+ Assessment 

• PDF Report Downloading of Student Report PDFs (Spring 2010 and Spring 201 1 results) 

• PDF Image Print of Individual Students Responses for Paper/Pencil (Spring 2010 and Spring 201 1 
results) 

• Group Download PDFs - Student Report PDFs and Image Prints of Individual Students Responses 
(Spring 2010 and Spring 201 1 PDFs) 

• PDF Generation of all online reports (Spring 2010 and Spring 201 1 results) 

• Online Rescore Request Form (Spring 201 1) 

• Corporation and Non-Public School level xsv GRT file download (Spring 2010 and Spring 201 1) 

• Corporation and Non-Public School level GRT file record layout download (Spring 2010 and Spring 
2011) ' . 

• Secure role-based access 

• Single sign -on 



'.t.-.A ft~L- '': t' . ''k c !jA y -"Indiana- Oidihe'.lteportn^ >■>■', " i v "V. Q 

• Online Reporting for ISTEP+ Assessment 

• PDF Report Downloading of Student Report PDFs (Spring 201 1 and Spring 2012 results) 

• PDF Image Print of Individual Students Responses for Paper/Pencil (Spring 201 1 and Spring 2012 
results) 

• Group Download PDFs - Student Report PDFs and Image Prints of Individual Students Responses 
(Spring 20 1 1 and Spring 20 1 2 PDFs) 

. . • PDF Generation of all online reports (Spring 201 1 and Spring 2012 results) 

• Online Rescore Request Form (Spring 2012) . 

• Corporation and Non-Public School level xsv GRT file download (Spring 201 1 and Spring 2012) 

• Corporation and Non-Public School level GRT file record layout download (Spring 201 1 and Spring 
2012) 

• Secure role-based access . 

• Single sign-on 

Included below is a more detailed specification of the solution and implementation timelines. Spring 201 1 
and Spring 2012 reporting will be consistent with deliverables and processes established for the Spring 2009 
and Spring 2010 administrations. 
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Included Datasets 

The datasets that will be included in the production solution for the Spring 20 1 1 are: 

• Spring 2011 ISTEP+ Assessment Grades 3-8 

• Spring 201 1 Student Report PDFs 

• Spring 201 1 Image Print s of Individual Students Responses 

Additionally, as a part of the Spring 201 1 INORS solution, the Spring 2010 ISTEP+ Assessment results data 
previously loaded into the INORS Will be available (see assumptions section below). 

The datasets that will be included in the production solution for the Spring 2012 are: 

• Spring 20 12 ISTEP+ Assessment Grades 3-8 

• Spring 2012 Student Report PDFs 

• Spring 20 1 2 Image Print s of Individual Students Responses 

Additionally, as a part of the Spring 2012 INORS solution, the Spring 201 1 ISTEP+ Assessment results data 
previously loaded into the INORS will be available (see assumptions section below). 

Implementation Timeline and Overview Spring 2011 and Soring 2012 

The Spring 201 1 and Spring 2012 implementations of INORS will consist of four (4) phases: 

• Planning 

• Development 

• Stabilize 

• Deploy 

Each phase has a defined purpose and scope and will act as a framework to ensure that your expectations are 
properly met as an updated version of this solution is introduced to your stakeholders. The dates listed below 
are for estimation purposes only. Upon final acceptance a detailed INORS project plan will be finalized and 
used to communicate dates, timelines, and milestones during this project. 

Defects that may be identified in INORS will be addressed as they arise. Requested changes to the 
functionality, or functionality/reports that are not agreed to during the planning phase for each year will be 
handled via a change order. CTB will work with the IDOE to ensure that all requests are prioritized and 
handled effectively as possible, while limiting time and resource requirements from all parties. It is important 
to have requests communicated in advance so any changes can be properly scoped, developed and tested early 
enough to avoid impacting the reporting window or INORS system functionality. 

Goals 

For the Spring 2011 and the Spring 2012 solutions, the goals are: 

• Deliver a fully functioning Online. Reporting System (ORS) to the Red-Team and Production 
environments for review and approval by the Indiana Department of Education (IDOE). 

• Provide online reporting and online report delivery in a timely and accurate manner to all 
stakeholders. 

Functionality 

This section describes die basic functionality that will be included within INORS. A more detailed discussion 
of the exact reports included is a later section. INORS is a browser-based system that is designed to deliver 
online reporting to the state of Indiana. The purpose of INORS is to enable educational leaders to make 
informed decisions about curriculum, school programs and facility needs, based on real data. INORS 
provides assessment data for the ISTEP+ test at a State-, Corp/Diocese-, School-, and Grade level for Indiana 
public and nonpublic schools. 
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Frequently Asked Questions (FAQ) 

A FAQ section is provided that can be used to answer basic questions that may need to be addressed. 
Help 

An online help section will provide users with searchable help that can be used to answer most usability and 
technology questions. The help system will additionally include tutorials and answers to basic questions. 
Help Desk support information is included in the Support section below. 

Manage Users 

The "Manage Users" section will allow administrative users the ability to manage all user accounts within . 
their authoritative domain. Administrative users have the ability to manage all descendant accounts with the 
hierarchy (i.e. a corporation administrator can manage corporation user accounts and both school 
administrative and user accounts). Hie only exception to this rule is the state administrator role. State 
administrators can manage all accounts (even other state administrative accounts) within the system. Manage 
users also includes the ability to create username and password sheets that can be distributed to INORS users. 

User Settings 

The user settings functionality is for individual users to manage their own account. From this interface users 
can change their password, update their email address and set the answer to their security/reminder question. 

Reports Area 

This section will include the available selection of reports that a user will have access to within INORS. A 
more detailed description of the reports is included below. 

Downloads Area 

This section will include all data files that are accessible for download by a particular user. 
Corporation level GRT downloads will be available within INORS to state and corporation level 
administrators. Non-Public school level GRT downloads will be available within INORS to state, 
corporation, and non-public school level administrators. The Spring 201 1 deployment will also contain the 
Student Individual Summary Reports and the Image Print of Individual Students Responses PDFs for both the 
Spring 2010 and Spring 201 1 test administrations. The Spring 2012 deployment will contain the Student 
Individual Summary Reports and the Image Print of Individual Students Responses PDFs for both the Spring 
201 1 and Spring 2012 test administrations. In addition, the Group Download option will continue to allow 
users to select ISR and/or image print PDFs for multiple students to be combined into one file to be 
downloaded and printed. 

Took Area 

The tools area provided within the INORS website can provide users with access and links to additional 
materials such as training and interpretive guides. During the final izat ion of scope and design, the INORS 
team will work with the IDOE to identify any electronic materials that should be made available through the 
INORS website. 

Informational Area 

At the bottom of the INORS website, an informational area exists that will display appropriate copyright, 
contractual and informational items. Legally, CTB must include copyright details, terms of use policy, online 
privacy policy, and applicable Children's Online Privacy Protection Act (COPPA) information. Additionally, 
customer support phone numbers, and email address will be included. 
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Header/Messaging Area 

At the top of the INORS website, a header and messaging area will be utilized to display messaging 
information. This area will include a small CTB/McGraw-Hill text and "The McGraw-Hill Companies 11 
image in the upper left and right corners respectively. Additionally, this area will include a "logged in as" 
notice which will indicate the name of user logged into INORS. This area will include the appropriate IDOE 
logo along with any messaging and naming that is required. 

Report Details 

Along the bottom of the menus, on the left side of INORS will be a Report Details section. The report details 
section is used to include informational material about the reports which could include hems such as scale 
score ranges, privacy notices, report legends, report purpose text, etc. The report details section is 
configurable for each report and will be defined during the planning stages. 

Backer Text/More Information 

A "more info" button or hyperlink will be included with each report and will contain the traditional backer 
text that would normally appear on the back of a printed report Backer text is customizable per report and 
will be collected through CTB. 

Print Report 

Each online report, as it is viewed on the screen, will be capable of rendering in a PDF format through the 
"Print Report 1 ' button or hyperlink. 

Report Header/User Control 

Each online report will include a user control that will enable users to navigate through the available options 
for the report based on the report and their level of access to the system. Options within the user control will 
vary by report, but the control will use "sticky" filters and maintain the prior selections while navigating 
through the reports where practical. 

Embargo Notice 

Based on feedback we have received, it may be necessary to include an embargo notice on reports based on 
what data has been officially released by the IDOE. It is our assumption that we will need to include this 
embargo notice and that we will need to easily and quickly be able to remove it from the reports, once the 
information has been released. The details of the embargo notice will be further finalized during the planning 
stages and incorporated into the design of INORS. 

Usage Reporting 

Basic usage reporting will allow administrators the ability to see when, if ever, a user last logged into INORS 
via the Manage Users functionality set 
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Homepage 

INORS Will continue to include the existing homepage. Any changes to the existing homepage will be 
finalized during the planning stage. Design or layout changes to the homepage will be considered out of 
scope. 

Acceptable Use Policy 

The existing acceptable use policy designed by the IDOE will continue to be included within die system. 
This Online Usage Agreement may be reset periodically by state administrators and is typically used to 
educate ORS users on their responsibility for securely managing data and reporting. 

PDF/Print-On-Dcmaod Reports 

Student Report PDFs will be available for download within the INORS. These PDF reports are made 
available to users based on their login credentials and are securely scoped into only their PDF reports. See the 
Reports section below for the detailed list of PDF reports that will be included. 

Secure, Scoped User Access 

All users will access the system by providing a valid usemame and password. All user accounts are directly 
linked to a level within the organizational hierarchy and will not allow users to navigate to information that is 
external to their access level. For example, all school level user accounts are linked directly with an 
individual school contained within the INORS. This allows the school user the ability to see all pertinent 
information about their school and only their school. State and System comparative numbers may be 
included for informational purposes within the report but the user will never have the ability to gain access to 
any other identifiable information about a student that is not within their school. 

Reports 

The following reports will be included within INORS for the Spring 201 1 ISTEP+ Assessment 



Spring 2011 INORS Reports 



Grade 


Report Name 


Description 


Mode 


TL 
Online 


TL 

Print on Demand (PDFs) 


3-8 


Student Report (STUR) 
Side 1 


Overall scale scores, passing levels, IPI scores 
by objective, information about areas of student 
strength and weakness 


Class 




X 


3-8 


Student Report (STUR) 
Side 2 


Writing Assessment information, points 
achieved by item, points possible, condition 
codes 


Class 




X 


3-8 


Student Image Print 


Scanned copy of student test booklet - Open 
Ended Responses 


Class 




X 


3-8 


Class Academic 
Standards Report (CPR) 


Lists students' mastery/non-mastery of 
standards 


Class 


X 




3-8 


Academic Standards 
Summary (ESSM) 


Compares 1 school to corporation - points 
possible, IPI at pass, mean no. correct, mean 
IPI, difference number mastery, percent 
mastery 


State 

(PDF) 

Corp 


X 
X 




3-8 


Group Academic 
Standards Summary 
(GESS) 


Same as Academic Standards Summary - 
compares multiple schools to corporation 


State 

(PDF) 

Corp 


X 
X 




3-8 


Proficiency Roster (PPR) 


Lists students and scale scores, performance 
levels, HOSS, LOSS, noVPCT at each - 
performance level 


School 
Class 


X 
X 




3-8 


Undetermined Status 
Roster (UDTR) 


Explains students* undetermined status 


School 


X 




3-8 


Disaggregation Summary 
Report (DISA) 


NoVPCT of students in each performance level 
by disaggregated group 


State 
(PDF) 
Corp 
School 


X 

X 
X 




3-6 


Class Proficiency 
Grouping Report (CPGR) 


Lists students in matrix by performance level 
achieved for E/LA, Mam, Science, and Social 
Studies 


Class 


X 
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3-8 


Writing Assessment 
Frequency Distribution 
(ASFD) Page 1 


Distribution of students by points obtained on 
Writing Assessment items 


State 
(PDF) 
Corp 
School 


X 

X 
X 




3-8 


Writing Assessment 
Frequency Distribution 
(ASFD) Page 2 


Distribution of students by condition codes on 
Writing Assessment items 


State 
(PDF) 
Corp 
School 


X 

X 
X 




3-8 


Academic Standards 
Frequency Distribution 
(STFD) 


Distribution of students by scale score for 
E/LA, Math, Science, and Social Studies 


State 

(PDF) 

Corp 


X 
X 




3-8 


Proficiency Performance 
Summary (SPPR) 


Lists students in matrix by performance level 
achieved for E/LA, Math (All), Science (grades 
4 and 6) and Social Studies (grades 5 and 7). 
Summary data includes HOSS, LOSS, mean 
scale score, and SD 


State 
(PDF) 
Corp 
School 


X 

X 
X 




3-8 


Rescore Request Form 
(RRF) 


Lists students that did not pass by content area 
and the items for which they did not receive 
full credit 


School 


X 




3-8 


GRT 


Electronic .csv Format Download (Note: Both 
Public Corporations and Non-Public Schools 
will be posted) 


Corp/ 
Diocese 
Non- 
public 
School 


X 
X 




3-8 


GRT File Layout 






X 





The following reports will be included within INORS for the Spring 2012 ISTEP+ Assessment. 

Spring 2012 INORS Reports 



Grade 


Report Name 


Description 


Mode 


TL 
Online 


TL 

Print on Demand (PDFs) 


3-8 


Student Report (STUR) 
Sidel 


Overall scale scores, passing levels, DPI scores 
by objective, information about areas of student 
strength and weakness 


Class 




X 


3-8 


Student Report (STUR) 
Side 2 


Writing Assessment information, points 
achieved by item, points possible, condition 
codes 


Class 




X 


3-8 


Student Image Print 


Scanned copy of student test booklet - Open 
Ended Responses 


Class 




X 


3-8 


Class Academic 
Standards Report (CPR) 


Lists students' mastery/non-mastery of 
standards 


Class 


X 




3-8 


Academic Standards 
Summary (ESSM) 


Compares 1 school to corporation - points 
possible, IPI at pass, mean no. correct, mean 
IPL difference number mastery, percent 
mastery 


State 

(PDF) 

Corp 


X 
X 




3-8 


Group Academic 
Standards Summary 
(GESS) 


Same as Academic Standards Summary - 
compares multiple schools to corporation 


State 

(PDF) 

Corp 


X 
X 




3-8 


Proficiency Roster (PPR) 


Lists students and scale scores, performance 
levels, HOSS, LOSS, noTPCT at each 
performance level 


School 
Class 


X 
X 




3-8 


Undetermined Status 
Roster (UDTR) 


Explains students' undetermined status 


School 


X 




3-8 


Disaggregation Summary 
Report (DISA) 


NoVPCT of students in each performance level 
by disaggregated group 


State 
(PDF) 
Corp 
School 


X 

X 
X 




3-6 


Class Proficiency 
Grouping Report (CPGR) 


Lists students in matrix by performance level 
achieved for E/LA, Math, Science, and Social 
Studies 


Class 


X 




3-8 


Writing Assessment 
Frequency Distribution 
(ASFD) Page I 


Distribution of students by points obtained on 
Writing Assessment items 


State 
(PDF) 
Corp 
School 


X 

X 
X 
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3-8 


Writing Assessment 
Frequency Distribution 
(ASFD) Page 2 


Distribution of students by condition codes on 
Writing Assessment items 


State 
(PDF) 
Corp 
School - 


X 

X 
X 




3-8 


Academic Standards 
Frequency Distribution 
(STFD) 


Distribution of students by scale score for 
E/LA, Math, Science, and Social Studies 


State 

(PDF) 

Corp 


X 
X 




3-8 


Proficiency Performance 
Summary (SPPR) 


Lists students in matrix by performance level 
achieved for E/LA, Math (All), Science (grades 
4 and 6) and Social Studies (grades 5 and 7). 
Summary data includes HOSS, LOSS, mean 
scale score, and SD 


State 
(PDF) 
Corp 
School 


X 

X 
X 




3-8 


Rescore Request Form 
(RRF) 


Lists students that did not pass by content area 
and the items for which they did not receive 
full credit 


School 


X 




3-8 


GRT 


Electronic .csv Format Download (Note: Both 
Public Corporations and Non-Public Schools 
will be posted) 


Corp/ 
Diocese 
Non- 
public 
School 


X 
X 




3-8 


GRT File Layout 






X 





Reporting Phases 

For Spring 201 land Spring 2012, interactive online reports and Student Report PDFs will initially be released 
for Phase 1 reporting with the data available after the Multiple-Choice testing and scoring window. An 
updated Phase 4 release of interactive online reports will take place after post Rescore data is complete. 



Support 

All customer and technical support will be handled by CTB/McGraw-HilPs customer support organization. 
Support requests can be made via a toll free 1-800 number or through email. INORS will include support 
contact information in the system, on the homepage and on all distributed material. 

Security 

CTB's ORS solutions use industry standard best practices to ensure the security of your data and solution. 
The ORS uses 128-bit Secure Sockets Layer (SSL) over HTTPS to ensure that no private data is intercepted 
during transmission. 128-bit SSL is used by online banks and credit card companies, so it is both secure and 
reliable. The INORS solution is hosted at The Standard & Poor's 55 Water building in downtown New York 
City at one of the most secure buildings in the city. The physical and network security of the 55 Water 
building supports some of the world's largest financial data applications and is used by Standard & Poor's, JD 
Power and Associates. 

Availability 

The INORS website will be available via an internet browser 24 hours a day, seven days a week. At times, it 
will be necessary to perform routine system maintenance and we will notify the 1DOE prior to commencing 
with any regularly scheduled maintenance. We attempt to schedule all maintenance during off peak hours 
(evenings and weekends) so as to provide as little disruption to regular service as possible. Should an 
unexpected system outage occur, the INORS team will use all practical resources to return INORS to a usable 
state as soon as reasonably possible. 
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Assumptions 

• For the Spring 201 1 and Spring 2012 solutions, online reports will include class level reporting. 
Interactive online reports will be available by grade at the class level and higher, as applicable. 

• Class breakdowns will be based upon the teacher of record within the data (GIS/GRT data). 

• Not all the reports will be made available in an online format Certain reports will continue to be 
delivered on paper (Student Label and Perfect Score report). See the Reports section above for a list 
of reports, which will be available online. 

• The Spring 2011 year of this program runs from July 1, 2010 to June 30, 2011. 

• The Spring 2012 year of this program runs from July 1, 201 1 to June 30, 2012. 

• This Work Plan Attachment section specifically addresses the Spring 201 1 reporting window. 

• Continuing in the Spring 201 1 and Spring 2012 reporting windows, PDF/Print-On-Demand Reports 
will continue to include the Student Report and Image Prints. 

• Longitudinal reporting costs are planned, expected or included in this Work Plan. 

• Current INORS functionality allows the IDOE to force a password reset when an employee of the 
state is no longer employed or at any time a State Administrator chooses to do so. 

Constraints " 

• All existing datasets will drive the online reporting. No new summarizations or aggregations (either 
pre-done or dynamic) will be available for INORS in Spring 201 1 and Spring 2012 reporting window. 

• State, corporation, school, and teacher level users will have access to the online reporting system. 

• Class and student level reporting will be available by the teacher of record, based upon GIS data (only 
the teacher on the pre-coded labels or bubbled in on a test booklet). 

• CTB will provide electronic materials via the INORS website that can distributed to individual end- 
users for self-training. . - 
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Attachment B 

The Parent Network and Individual Student Report 
Spring 2011 and Spring 2012 Work Plan 

Beginning in Fall 2008, CTB Individual Student Reports and Indiana Parent Network were added to the 
ISTEP+ assessment contract. Individual Student Reports will be provided to parents in print and to educators 
and parents electronically as PDFs. The Indiana Parent Network provides parents with student ISTEP+ 
results and guidance for working with students 

Deliverables ... 

• Provide Indiana parents with Student Reports and resources in both print and online 

• Provide Indiana parents with a PDF copy of their child's Applied Skills image print 

The Individ aal Student Reports 

1 . Multi-subject Student Reports (optimized for PDF distribution) 

2. Overall, standard-level, and item-level results 

3. Content includes introductory text, interpretation guidance, resources 

4. Provided in 3 formats 

a. PDFs for parents on Parent Network 

b. PDFs for educators - collated by school and corporations., for electronic download on educator 
site (INORS) 

c. Printed version of Student Reports - I B&W (double-sided) copy shipped to corporations., 
collated by school (public corporations), shipped to schools (non-public and charter schools) 

5. Student Reports printed in two separate operational runs - the first run will contain all final student data 
and the second contain any students whose scores have been changed during the rescore period. Corrected 
Student Reports are also posted on Parent Network and INORS. 

6. Tracking of packages provided 

Student Open-Ended Response Images (Image Print) on Parent Network 

1 . Parent Network hosts scanned "Image Print" files with images of student work 

a. File will be in PDF format (assumption is -500K per assessment year) 

i. Each file represents a single student's result images (all subjects in one file) 

ii. Assume naming convention based on Applied Skills barcode, grade and subject 

iii. Assume files provided in file directory collated by grade, subject, and school 

b. Files transmitted using a secure FTP or secure shared server after reporting window 

c. Image Print PDFs provided on the Parent Network website as PDF download 

d. Parent Network will need to re-launch to include Image Print PDFs in addition to ISR PDFs (one 
■ per student for both) 

e. Image Prints available on the Parent Network for all students tested in Grades 3-8 (including 
those students who test on-line). 

Spanish Translation 

1. Translation Guides to the Individual Student Report (In English and Spanish) in PDF, not printed 

2. Translation of the Parent Letter sent to parents into Spanish 

Parent Network 
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1 . Online website for secure access by families of ISTEP+ assessment results and download of Student 
Reports 

a. Grades and subjects: 

i. E/LA and Math: Grades 3-8 

1. Administered: SI 1 and S12 

ii. Science: Grades 4 and 6 

1. Administered: SI 1 and S 12 

iii. Social Studies: Grades 5 and 7 

1. Administered: SllandS12 
Launch after state release of results to corps, and schools 
, Invitation letters (login letters) generated for parents (one for each student based upon pre-id 
file) and delivered to schools (non-public and charter) and corps (public schools). 

i. Parent Network provides functionality for schools to generate invitation codes for 
students who were not in the pre-id file, but have taken the assessment 
Graphic display of score results at overall and standard level 
PDF download of CTB Student Report on site for parents 

i. Updates made to student records whose scores have changed after the rescore period 
Links to Indiana instructional resources and released items 
PN will re-launch after each administration to include data from the current test 
administration as well as previous test administrations. 

i. For Spring 20 1 1 CTB will begin to show 2 test administrations (current and Spring 
2010). For Spring 2012, CTB will show 2 test administrations (current and Spring 
201 1) 

ii. All other test data prior to the test administrations mentioned above will not be 
loaded into the PN within the scope of the contract 

h. Display student's OE responses (image print) in PDF • 

i. Customer Support provide via phone and email 

i. Hours of operations will be 10am-6pm EST 

ii. Response Time for Severity One issues will be within 24 hours of issue arising 



b. 
c. 



d. 
e. 

f. 
g- 



Tasks 



r Phase/Milestones - 


-,. *■ \ 




.: Dates* v 


Student Report 








Requirements 








Project Initiation 








Kick-Off 






TBD 


Level-OPlan 






TBD 


Client Requirement 






TBD 


Mockups/Prototypes 








PNDemo 








Client Review and Sign off 








Project Requirements 






TBD 


Functional requirements 








Data requirements 








Collation Requirements 








Cycle Planning (Design and 






TBD 
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Analysis) 








Development, Unit Testing — ISR 








Parent Network 






TBD 


ion 

ISR 






TBD 


QA Testing (Mock data, Mini data) 








Parent Network 






TBD 


ICD 

ISR 






TBD 


Operations 








rre- iu 






■rnrv 

IdU 


ISR 






TBD 


Parent Network 






TBD 


Refresh 








PN Rescore Launch 






TBD 


ISR PDF's received to TL 






TBD 


ISR Printed and delivered 






TBD 



Assomntions: 

• ISR PDF's will be used for both Parent Network and INORS. 

• Item level data, full item release, and record of student work "image-print" are utilized for Parent 
Network release 

Files of associated barcodes with pre-codes assigned so that the barcodes are utilized for matching with 
GRT. 

Schedule Assumptions 

• Internal CTB Red Team Review of report output . 

• A final review and sign off of Parent Network and Student Reports by IDOE will occur prior to 
release/ before operations begins 

• o CTB will work with the IDOE to define final Red Team Review and release schedule 

Parent Network will launch on a date mutually agree-upon between IDOE and 
CTB in relation to the seven-day turnaround schedule. In 2009 Parent Network 
launched a week after the INORS launce, which allowed educators to become 
familiar with student results prior to parent review. 

Security 

CTB has many years of experience in handling millions of students' standardized assessment data without 
any known security breach. The website is managed by McGraw-Hill's Global Operations and Infrastructure 
Division, the group that also hosts Standard and Poors, and Platts. The data center has extensive backups and 
recovery procedures, and was operational through both September 1 1 th and the New York City black out. 
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Printing/Shipping 
Shipment Detail 

• Parent Network Login Letters will be shipped to the -556 corporations. They will be printed in black' 
and white, single impression. 

• Hie Student Reports will be printed at a vendor acceptable to the IDOE and shipped with the Student 
Labels, ancillaries, and cover memo. 

• Student Reports - One (double-sided) copy of the Student Reports created and sent to schools. 
Student Reports will be printed in black and white. (See Report Requirements for more 
specifications) 

Report Requirements 

Parent Network and Student Reports 

1. Students = Grades 3-8 

a. 6 Grades = 552K students 

b. Based on Grades 3-8 

2. Product 

a. Parent Network 

i Parent Network hosts PDFs of reports in addition to images 

ii Components 

a Login generation based on Pre- ID files 

b Dissemination of Logins 

o Letter to Families with login and password information 
o School Admin codes to be able to access Parent Network 

c Application 

o Custom for IN 

■ Introductory letter (landing page) 

■ IN logo 

■ IN signature 

■ IN content and translation guides 

■ FAQ 
d Student scores 

o graphic display of the scale score 
o achievement level 
o standard measurement of error range 
o PLUS new topic-level display 
o PLUS PDF download 
o interpretive guidance 
o No item level results on parent site 
e Content Resources 

o Grade-banded state and national resources 

■ Based on grade of users, provide 3-5 links and screenshots of 
web resources 
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o Standards-based resources 

■ The description of standards in parent friendly language 
including real world example 

■ Link to IN web resource that describes standards, 
benchmarks in detail 

■ Link to IN web resource that has activities 
o Assessment Literacy 

■ Provides links to IN released items 

■ Test information 

f Image Print 

o Combined PDF for all tested subjects 

o Includes average of 10 images of student responses (from Applied 

Skills portion of each subject test) 
o Closely connected to links of released items 
o Direct linkage from item-level results to item response not possible 

due to lack of PDF indexing 
Student Reports - Providing as PDFs and in print 

a Scores include overall, subscore and item results for 3-4 subjects 
b Format - single-subject 
o 2 versions: 

■ PDFs designed for black and white printing 

■ Black and white printed copy 

o Single shell for design, no actual shells used in printing 
o 2 page (8 Vi x 1 1 ) report, double-sided 
o Single-subject reports 

• . All subjects: 3-8: R, M, SC, SS 

• Alternate: 3-8: R, M; alternating S, SS 
c Components 

o Front page 

■ Intro letter 

■ Gold box 

■ Overall scale scores - traffic light display 

• Pass+, Pass, Did not Pass 

• Narrative text with score and performance level 

■ Performance level descriptors 

■ Descriptive text integrated into letter and along bottom 
" Important general IN resources provided on front 

o Backpage 

■ Introduction text to standard and item level results 

■ Standard (topic) level results 

■ Graphic display showing pass/not pass 

• data reported by each standard 

o test item types 
o student IPI score (number) 
o Passing IPI score 
o Corporation average 
o State average 
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• Item level results 

• item number 

• standard tested 

• page number on test booklet 

• points earned 

• points possible 

■ Resources - provided in all grades with some subject- 
specific resources, grade-banded 

d Delivery 

o Student PDFs parsed as individual files for hosting on Parent 
Network 

o Educator-collated PDFs collated into educator-batches by 
corporations, and schools for download on INORS site 
tv Spanish translation 

a Translation Guide for of the Student Report 

3. Assessment Results 

a. Total subjects/grade = 2 or 3 (see chart below) 



Grade 


Reading/LA 


Math 


Science 


Social Studies 


Total Subjects 


3 


X 


X 






2 


4 


X 


X 


X 




3 


5 


X 


X 




X 


3 


6 


X 


X 


X 




3 


7 


X 


X 




X 


3 


8 


X 


X 






2 



Performance 

Performance Levels = 3 (Did not pass, Pass, Pass+) 
ii Standard level results = Student IPI, Passing IPI, and State Average 
fv Data 

a Results for ISTEP+ processed at same time 

a Student Results will be provided using the GRT 

b IN has unique student IDs called STN 

c The applied skills file will have the barcodes assigned 

d Assessment results processed by state completion 

e See TAT schedule above for specific processing steps 
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Attachment C 

ISTEP+ Online Assessments 
Spring 2011 and Spring 2012 Work Plan 

CTB will provide ISTEP+ both paper and online assessments for English/Language Arts and Mathematics in 
Grades 3-8, Science in Grades 4 and 6, and Social Studies in Grades 5 and 7 for both Spring 201 1 and 
Spring 2012 administrations. 

Background - Year 2009 Administration Summary 

In October 2008, a pilot test was conducted with a test combining constructed response as well as multiple 
choice and gridded hems. A small number of schools ( 1 0) were successfully introduced to CTB online 
summative testing. As a result it was determined by the IDOE that Indiana students may not be sufficiently 
prepared for applied skills testing online. A decision was made not to do applied skills in spring 2009 as well 
as in spring 20 1 0. For the Spring 2009 a number of 1 1 ,255 students took the Multiple-Choice test online 
(about 3% of the 3-8 student population). 

The following features were implemented in OAS in 2009 to support of the ISTEP+ program: 
Program Status Reporting 
Invalidation at subtest level 

Online Test Administration Parameters 

Grades and test that may participate in online testing listed in the table below, Applied Skills for ELA and 
Social Studies and Multiple-Choice for all subject areas. 

Applied Skills refers to tests including CR, ER, and Writing Prompts and Multiple-Choice refers to tests 
including Selective Response (Multiple-Choice) and Gridded Response items only. 



Applied Skills 



{Grade? 


E/LA 


^MathT 


; Science*. 


r S6ciaI ! Studie's;' • 


3 











4 


m 








5 


m 









6 











7 












8 
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Multiple-Choice 



iGrade;; 




IMathl 


iSciehcel 


m, SocialStadies^ 


3 












4 













5 













6 













7 













8 















^S^rin^011^012tt^»^j 


Applied Skills Dates 


TBD 


Applied Skills Practice Test Administration 
Dates 


TBD 


Number of schools online 


TBD 


Number of students 


TBD 


Multiple-Choice Dates 


Last week April- First week 
May 


Multiple-Choice Practice Test 
Administration Dates 


Minimum 5 days prior to 
Multiple-Choice 


Number of schools online 


458 (20% of total 2290, est) 


Number of students 


20% - approx 102,000 



Constraints 

1 . The CTB online testing capacity set aside for IN is for up to 30,000 simultaneous non accommodated 
students. Up to 5 % of the students testing online in Indiana are expected to use the Text Reader 
accommodation. 

2. School staff generally not available during summer (approximately June 1 through August 1). This could 
limit data collection as well as site certification. 

3. Participation of schools to online testing is voluntary. 

4. Participation is contingent upon schools/corporations meeting minimum hardware and software and 
network requirements. 

5. Site Readiness reviews can only be conducted during school operating days and hours. 

6. Scheduled holidays, institute days, staff development days, school and/or corporation mandated tasks may 
preclude staff availability for Site Readiness reviews as well as training for OAS. 
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Assam prions 

1 . OAS upgrades requested by DOE for features not currently available in the system will he handled as 
change requests. 

2. It is assumed that only a single reference card is required for Grade 6-8. 

3. Site Readiness reviews are assumed to take 4 hours each and with 2 visits accomplished in a single day. 

4. Most Site Readiness preparation will be done remotely with an onsite support for 10% of all sites per year 
(45 schools sites will be visited by CTB). 

5. School sites will need to download the student test client as a part of the Site Readiness work. 1 0% of the 
buildings as indicated in Online Test Administration Parameters above will be able to have on-site 
assistance with this work as a part of the onsite Site Readiness visits. 

6. It is assumed that the OAS must provide the capacity to test approximately 1 00,000 students per test 
administration in Indiana. A change request must be processed before December 2009 if more students 
need to test online. 

7. The OAS Implementation Services Manager will plan and visit a number of school sites during the test 
administration. The visits will be conducted during the peak testing period for 2-4 days and will include 
approximately 2 school sites per day. 

8. It is assumed that all accommodations testing (including Large Print, Braille, and CD's) delivered in a 
paper and pencil mode for the duration of this RFP response (through Spring of 20 1 2) are matched to the 
extent appropriate by equivalent accommodations currently available in OAS such as large font and text 
reader. 

9. It is assumed that slight look and feel variations are expected when the paper pencil content is transferred 
online. 

Operational Online Administration Spring 2010 

For Spring 2010 administration, CTB will provide Online Assessment services to test 20% of the student 
population in grades 3 to 8 as stated in the Online Test Administration Parameters section above for 
Applied Skills and Multiple-Choice. The key services and related deliverables are described in the 
following sections. 

CTB will follow the OAS Implementation methodology, that provides for constant communication with 
IDOE on the planning and definition of tasks and activities. It includes extensive internal and client 
reviews of all custom deliverables leading to their approval to very high standards and full customer 
satisfaction. As part of this methodology the following key services and deliverables are included in scope 
and described in the following sections: 

• OAS Implementation Project Management 

• Content Production Support 

• Client Specific OAS Configuration 

• - Data Provisioning: Student Pre Identification Data and OAS User Data 

• Training and Documentation 

• Implementation Services: Site Survey, Site Readiness and Certification, and Test Administration 
Support 



Exhibit A, Page 43 of 66 



Contract No. 05688-KG08-000 

OAS Implementation Project Management 

Standard project management processes are employed for planning, executing, monitoring and control 
and closing the project. A detailed implementation plan and schedule will be developed for each year of . 
the contract. Hie project execution will be closely monitored to ensure the project objectives are met on 
time and within budget. An integrated change management process will be used to ensure that negative 
impacts of changes during the life of the project are reduced. The project risks will be analyzed 
periodically and mitigation plans will be identified and risks are managed to closure. 

Client Specific OAS Configuration 

OAS is a configurable system that allows setting certain features as needed by the customer without any 

software code changes. 

These configurations may include: 

• organization hierarchy definition and labels names 

• user roles and assignments 

• student demographic labels and values, student pre- identification data protection 

• accommodations settings for background and font color changes controlled by the test 
administrator, calculator, text reader, un-timed test 

• product acknowledgment page 

The configurable elements in OAS will be reviewed in detail with DOE and as needed screen mock-ups 
will be provided to confirm the settings. A business and configuration requirements will be created and 
will be used to apply the configuration settings to OAS prior to the test administration. 

Content Production 

For Content Production OAS Implementations and Development will provide support to develop the 
content framework, promote the content to the various review environments and provide the content 
review web based tools. This support will be provided to produce for each administration year one 
content form per grade per subject as well as two - three practice tests forms per administration year. 
Refer to the Publishing scope for details regarding the content scope and forms design. 

Data Provisioning 

Data provisioning is done to preload the students* data into the system. 

For client acceptance purposes a simulated corporation and school will be identified and created in the 
acceptance and production environment. The data for user acceptance this will be exported for scoring 
and reporting and provided along with the paper and pencil live data after the test administration. 



Exhibit A, Page 44 of 66 



Contract No. 05688-KG08-000 



Student Pre-Identification Data 

OAS will make use of the Spring 201 1 and 2012 respectively student barcode files to select and load 
the students for the grades indicated in Online Test Administrators Parameters. DOE will review the 
student data in OAS prior to opening the system for general schools use. The student preloaded data 
can be modified in OAS and the users can also create new student data. Screen shots will be provided 
to confirm the demographic labels, positions and other graphic user interface aspects (e.g. using 
checkboxes, dropdown lists, and radio buttons for values selection). 

As in previous years, the pre-id data can not be modified in OAS and the RES updates will prevail. 
Due to the timing of the RES updates, those would not be applied until just prior to the data extract 
from OAS. 

OAS User Data 

Pre-existing CTB SSO user accounts are enabled in OAS for every staff member identified during the 
school selection process and according to their specific role. New accounts will be created as needed 
as well. These OAS accounts are used to access the system by the technical staff, test administrator 
staff and test proctors and perform the activities permitted for each user role. 

Training 

Training materials will be provided to the Indiana staff to ensure a successful implementation. All the 
training materials will be provided in electronic form and posted online at <www.ctb.com/istep>. 
Aside from the training materials described in the table below, in 2009 CTB developed at the IDOE's 
request also a pre-recorded practice test for students. The pre-recorded practice test will be again provided 
. unchanged in 201 1 - 2012. 
The following training sessions will be provided: 











, ©UlUuMUfl 


^^HTcs tlVtl min is I r:i t i o n;(!)^rvjc"^^H 




ffiEflffuTB 
Bhiiflhflmiife 


Corp Administrators (N=l 00) 
State Administrators (N=10) 


Corporation IT Specialists 
(N=50) 






ofltibs 

ffiVm 


Introduction to the Summative Assessment 
Program: Overview of OAS features and 
functions. Training in Administrator tasks 
and responsibilities. 


Introduction to the OAS system; 
Overview of technology architecture 
and implementation procedures. Set- 
up of Administrator and Student 
Work Stations 




Hands-On; Online conducted on-site with 
Webinar make-up; Demonstration and oral 
presentation Train-the-Trainer. 


Online - Webinar only; 
Demonstration and oral presentation. 
Train-the-Trainer 




ItaSfliDQ 




5 half day sessions in S days 
(2.5 hrs each) 


3 half day sessions in 3 days 
(2 hrs) 




sflaaSfcD 


EBQX3F 

- *■ ■ 


20 


20 


" 


utngflm 




Up to 2 (Assumes 2 teams of trainers) 


Up to 2 (Assumes 1 team of trainers) 




Spring 2009, 2010 


Spring 2009, 2010 
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PowerPoint presentations, OAS User's 
Guide, ISTEP+ Online TEM, Handouts 
(a Digital OAS Admin Toolkit which 
can be downloaded off the 
<wwwxtb.com/isteD> site) 


PowerPoint presentations, OAS 
User's Guide, ISTEP+ Online TAM, 
Handouts Administrative Checklist; 
Technology Checklist. 




All Material posted Online 

Make-up web-ex, performed and posted 

online 


All Matenal posted Online 
Make-up web-ex, performed and 
posted online 




3-5 regional Training sites throughout 
the state will be provided by Indiana DOE 
(depending on participation) 


Webinar 



Documentation 

OAS includes standard online help in the system, standard user guide and installation guide. These guides 
are distributed in PDF format. 

The custom Indiana OAS Quick Start Guide, which was produced in 2009 will be updated as required and 
re-published in 201 1-2012. The Indiana TEM will include instructions about conducting the testing for 
Test Examiners. 

Implementation Services 

As part of CTB's implementation services, CTB's technology surveys will be distributed to identify the 
technology at each school considering participation in the ISTEP+ Online. The sites will be grouped by 
their ability to meet requirements (Fully Meets, Conditionally Meets, and Does Not Meet). Support for 
sites to help them achieve readiness would include documentation, emails, teleconferences and she visits, 
as appropriate. A report similar to the one provided in 2009 of all sites' readiness will be delivered to the 
State.. 

Site Readiness 

Based on our experience, paying close attention to these final steps prior to testing is critical in order to 
provide a reliable testing environment and prevent issues during the test administration. 

At the time the training is completed for the schools that are preparing to test, the supporting 
documentation for the system will have been provided to the participating schools. Hie schools' technical 
staff will be involved to configure and confirm that the workstation set-ups are completed and the 
corporation technical staff should be involved with school staff to ensure network configuration . 
requirements are fulfilled. Checklists are provided to guide the school and corporation staff during these 
activities. At the schools visited by CTB's technical staff, these activities will be performed by the school 
and corporation staff respectively, as monitored and assisted by CTB's field engineers. 

Users who are setting up student workstations can log into OAS, where utilities will automatically detect 
if correct versions of Java and Flash are on the workstation. School technical staff would download 
upgrades as necessary. 
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At this time, the encrypted tests must be downloaded and pre-positioned on the students' workstations. 
Site Certification practice test sessions should be scheduled. Each site must confirm to CTB that all the 
Site Readiness activities were successfully completed. Only when this final step has been completed can 
the online assessments begin for that site. 

Site Readiness Plan 

Based on data provided during the proposal preparation, there are 2290 buildings within the state that 
service at least one grade in the 3 through 8 grade span. 

For the scope of each of the administration years it is assumed that in order to reach 20% of the Indiana 
student population (100,000 students) up to 458 schools will be participating and CTB will visit up to 45 
sites to assist with technical site readiness for the online testing. If required more visits can be provided 
for additional charges.. 

Registration for Spring Online testing 

• CTB prepares the Site Technology Survey and Registration form 

• CTB distributes the electronic Site Survey to all interested sites. This survey is to be completed by 
each site's technology coordinator and returned to CTB within 10 business days of receipt. 

• The results will be used to adequately estimate which schools meet the OAS Configuration 
Requirements and are pre-qualified to participate in the online testing program. 

• CTB follows up with sites. Should any sites that indicate are interested to test online fail to send 
complete/accurate information, CTB will schedule phone conferences with the site and/or corporation 
technology coordinators in order to clarify and rectify any outstanding issues with the survey. 

• Should CTB determine that the site is not (or may not) be ready for OAS testing, in its opinion, the 
site will be advised if their equipment needs to be upgraded and plan to test only after mat upgrade is 
completed or an onsite visit may be scheduled to prepare the site for testing. 

• The OAS Implementation Services Manager will be responsible for delivering the following: 

■ School Technical Capabilities Report 

■ . Recommendations for schools not fully meeting online testing requirements 

■ List of Registered Schools for online testing 

■ List of Contacts that require accounts for online testing; we will add one tech 
coordinator and one test coordinator per school and one tech coordinator and one test 
coordinator per corporation. Corporation staff will be trained how to add users 



Test Administration Product Support 

Product Support for OAS users will be provided during me customer's business hours 7:00 am - 5:00 pm 
EST. OAS Product Support will provide real time support to each school that is administering the online 
test. Product Support will answer any questions and work with users in resolving any issues regarding the 
testing process, system procedures, error conditions, student testing status and system status. A weekly 
administration report will be provided. 

OAS Product Support is backed by CTB's network, development and technology support teams so that all 
customer calls can be handled expeditiously. All customer calls are logged and tracked to enable analysis 
of call handling and problem resolution statistics. .This results in increased process and system 
improvement. 
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Scoring and Reporting 

The'student test data will be provided to other systems in CTB for Scoring and Reporting along with the 
paper and pencil tests. The details for these activities are described in detail in the Scoring Scope and 
Reporting Scope documents and are not part of the present document. 

Out of Scope 

1 . OAS Manipulatives: any maniputatives not currently available (such as coin or shapes) are not in scope. 

2. Training: For training related travel, CTB will cover the travel and accommodation costs for CTB staff. 
CTB will not be responsible for the cost of the educators' travel to regional training sites. 

3. Pass-fail raw scores visible to students at the end of the test 

Security 

. OAS employs industry-standard best practices to ensure security of test content, the testing experience, 
and the student reporting data. The system operates in Web-based mode, with all communication 
between the OAS server and the student workstation over HTTPS. 

The test content is securely stored in an encrypted object bank using 256-bit encryption. Encryption keys 
are not retrieved from the server until the student successfully logs into a test, and the keys are only 
placed in temporary memory. Hie encrypted test items stored on the hard drive are only decrypted at the 
moment a student requests the test item. The item remains encrypted at all other times. 

The testing experience is provided in a browser window that is locked-down on the student desktop. The 
browser window uses the full screen, overlaying any task bars or other control elements on the computer 
desktop, and cannot be minimized. This prevents students from accessing anything other than the test 
delivered by the browser. Students only have access to the testing experience with a proper login ID and 
password, and these are unique for every test session to which the student is assigned. Cheating is also 
discouraged by including features such as random test form assignment. Further, should a student be 
suspected of a testing impropriety, the administrative interface allows a student's test to be invalidated 
(and re-validated) at the click of a button. 

Access to the student data is strictly controlled through a system of user roles and assignments in the 
organizational hierarchy. Users may only have access to the data and functionality appropriate to their . 
role, and only for the members of the organization to which they are assigned. 

Test administrators overseeing the students during testing can monitor student test status in real time 
using the OAS administrative software, and (depending on each organization's rules and procedures) can 
invalidate test results for students suspected of cheating, without losing the student's results or disrupting 
the testing process. Security is maintained in the OAS administrative software by means of hierarchy 
controls at various access levels, using a structure of limited permissions and login passwords. 
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To ensure business continuity OAS uses redundant fail over resources at the database, application server 
and web server layers. OAS is hosted by Equinix IBX centers which set the industry standard for secure 
and reliable collocation services. Equinix has a history of delivering robust and reliable data centers in 
the industry. OAS is hosted by McGraw-Hill's Global Operations and Infrastructure Division, the group 
that also hosts Business Week, Standard and Poors, and Platts. The data center has extensive backups and 
recovery procedures, and was operational through both the 9/1 1 tragedy and the New York City blackout. 
OAS is prepared also for situations when the entire Data Center is affected by natural disasters such as 
earthquakes or hurricanes. A disaster recovery plan and resources are in place to ensure the business 
continuity in case of catastrophic event 



Project Deliverables 



Deliverable: 


k ,*-3v; > r *Descnptaom ,T _i fi ^ 


Produced By 


^Staf^ol^ ^natures} - 


Project 
Schedule 


A set of detailed milestones that 
identify when the client 
resources are required for 
reviews or other activities. 


Program 

Schedule 

Analyst 


DOE and Program 
Management 


Content 
Requirements 


Documentation on framework 
code, commodity code, 
assessment title, deliverable unit 
titles, forms, level/grade, time 
limit, product name, number of 
subtests. 


Publishing 


DOE and Program 
Management 


Practice Test 
Content 


Set of custom test items to be 
used for student practice testing 


Publishing 


DOE and Program 
Management 


Operational 
Test Content 


Set of test items to be used for 
operational testing. 


Publishing 


DOE and Program 
Management 


Technology 

schools 

selection 


A list of schools that nave the 
technology and infrastructure to 
test online 


Impl Mgr 


DOE and Program 
Management 


Weekly Call 

Tracking 

Report 


Excel workbook with detail 
statistics on customer calls and 
corresponding quality metrics 
including trend 
analysis/summary. 


Cust Support 


DOE and Program 
Management 


Technical 
Training Slides 


Presentation slides to be used for 
Technical Training. 


Training 
Mgr/ 

Contractor 


DOE and Program 
Management 


Administrator 
Training Slides 


Presentation slides to be used for 
Administrator Training. 


Training 
Mgr/ 

Contractor 


DOE and Program 
Management 


Standard OAS 

Installation 

Guide 


Standard Installation Guide and 
Checklists 


OAS Tech . 
Writer 


NA 


Standard OAS 
User Guides 


Standard reference document for 
users who will be responsible for 


OAS Tech 
Writer 


NA 



Exhibit A, Page 49 of 66 



Contract No. 05688rKG08-000 













managing users, adding or 
modifying student data, 
scheduling tests, and proctoring 
tests. 






OAS Quick 
Start Guide 


Short guide on main functions 
described in the system. 


OAS Tech 
Writer 


DOE and Program 
Management 


TEM - online 
section 


Test Examiners Manual provided 
in PDF format. 


Publishing 


DOE and Program 
Management 


OAS Business 
and 

Configuration 
Requirements 


ISTEP+ Business and 
Configuration requirements. 


CTB 

Requirements 
Analyst 


DOE and Program 
Management 


Student Data 
Load (PrelD 
Load) 


Student data, including 
demographics. Typically, data is 
from pre-code. 


ETL Dev 


Product Support, 
optional DOE 


User Accounts 


User ids and passwords to allow 
state and school officials OAS 
system access, limited by the 
user's role (administrator, 
coordinator, accommodations 
coordinator, proctor) 


Cust Support 


NA 



Critical Success Factors 



Critical Success Factors 


U~ ''f&'-.^l&s* . V fei"^ ^ ; /--.'-V ; 7- , Snc^ea Definitidni- ' : -i^y 


Project Schedule 


Delivery of the following components minimum 2 weeks prior to the test: 
o OAS platform configured for Indiana 
o OAS loaded student barcode data 
o ISTEP+ Online Practice and Operational Content forms 
o Test Administrator's Manual 
o Technical and User Training 

Sites registration for online testing by end of November 

Site Certification activities completed at least 3 days prior to the practice 

testing 

Approvals for any deliverables are received in max 5 days from DOE. 
Online testing is completed in time to facilitate the 7 days reports TAT 


Project Scope 


Delivery of the following critical components: 

o Schools selection based on technology criteria and online 

testing registration response 
o OAS configured for Indiana 
o ISTEP+ Online and Practice Content forms 
o -Test Administration Manual 
o Technical and User Training materials including the 

Quick Start Guide 
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Critical Succ^ Factors Si 




Project Quality 


The delivery of this project within CTB's metrics is a critical success 
factor for this project. These metrics include: 

o Schedule Variance - Non-critical Components within a 

+/- 20% variance from the base lined project schedule, 
o Schedule Variance - Critical Components - Zero 

variance for components on the operational test 

administration critical path components, 
o Effort Variance - Delivery of the project within a +/- 5% 

variance from the base lined definitive estimates, 
o Post-Production Defects - No priority 1 and 2 defects 

post-production. 


Resources 


IDOE representative is available to facilitate response and resolution to 
State related issues/decisions within the agreed time frame 

IDOE and schools staff will be available and responsive to CTB's 
inquiries for site survey, certification and readiness. 

CTB is provided access to the schools and corporation facilities in order 
to perform site certification related activities. 



Project Acceptance 

. Prior to the release of the system to the participating schools, CTB will provide DOE access to the system 
in a User Acceptance Testing environment for acceptance and sign off for the Content, OAS Platform and 
configurations for Indiana, as well as a review and sign off of the loaded schools hierarchy and student 
data. 



Every effort is made to ensure that at this stage no critical or significant issues are present. It is expected 
that any non critical issues encountered at this time will be addressed by including further information in 
the training materials while a long term solution will be provided in the following administration. 
The User Acceptance Testing should be conducted in the final production environment in a UAT 
dedicated district and schools. The users will take tests as students and the results will be provide at the 
end of the testing along with all other student results. 

Content Acceptance 

The final Practice and Operational content user acceptance review is done using the OAS Test client in a 
User Acceptance Testing environment that replicates the OAS production environment At the time of 
this final review the content has already undergone several reviews and all critical issues were detected 
and addressed. At this final review the content should be used during test taking as students and only 
major issues if any may be addressed if possible in the short time frame remaining until the test 
administration. Any minor issues identified at this time will be addressed during the following 
administration. 
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OAS Platform and Configuration Acceptance 

The OAS platform and the configurations specific to Indiana will be provided for a final client review and 
the following criteria will be used for Acceptance. 

The OAS platform and configuration are deemed accepted when the Test Administration System and the 
Test Client configurations are set according to the ISTEP+ business and configuration requirements for 
data hierarchy configuration, user set-up, student demographics and accommodations. 

Student Data Acceptance Criteria 

The client will verify that the number of students expected to load have been loaded and will do a spot 
check of the student demographics data for the students included is loaded in OAS. 

Localized student data issues or missing student data affecting less than five students in a school location 
will be addressed by the client either correcting or creating new students data using the OAS system. 
A RES update will be applied before the online testing students' data is merged with paper and pencil 
testers for scoring and reporting. 

Project Key Milestones 





?DeMriptidn>%-., - : ;i^H^-i& : .^iw.^ 


?DueI>atel^v;; 


Sites Selection 


School sites selected 


November 


Indiana Acceptance 


Acceptance of the project deliverables complete and 
can be released to the schools and corporations. 


March 


Training . 


Training sessions completed 


March -April 


Schools and Corporation 
staff login 


Indiana staff receives OAS login username and 
passwords 


Late March 


School Sites Readiness 


School sites are ready for testing 


Min 5 days 
prior to 

operational test 


Practice Test 
Administration Multiple- 
Choice 


Students take the Practice test 


Min 5 days 
prior to 

operational test 


Test Administration 
Multiple-Choice 


Students take the test 


Last week of 
April - First 
week of May 


Test Administration 
Complete . . 


Last day of testing 


First week of 
May 



Technology Requirements 

The technical requirements listed below are based on the OAS 7 Installation Guide. In order to 
participate in the ISTEP+ Online, schools must meet these requirements. 

These requirements may change until the time of the test administration every year as it is expected that 
there will be new application upgrades. The upgrades are carefully planned to keep the system updated 
with recent technology advances and to facilitate more extensive participation in online testing. As 
more information becomes available, CTB will inform DOE, and as appropriate, corporations and 
schools, about changes in requirements. 
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REQUIRED COMPUTER EQUIPMENT 

Each computer must meet these minimum specifications: 

• Platforms: 

■ Windows PC with 1 .3 GHz CPU and Windows XP Service Pack 3 or greater, Windows 
2000 Service Pack 4 or greater, or Vista Service Pack 1 or greater OR 

Intel Macintosh with 1 .8 GHz CPU and 1 0.4.8 OS or OS 1 0.5.4 

■ Linux (Suse, Ubuntu, Rehat) 

• Hard disk space: 200 MB available disk space for object bank 

• Memory: 512 MB RAM 

• Internet access: 1 Omb Ethernet connection or equivalent wireless connection 

• Bandwidth: 1 .5 Mbps upload and download- Tl line, high-speed cable or high-speed DSL for 
every lab with 50-100 concurrent users 

• Software: 

■ Java Runtime Environment (JRE): Java 5.0, Update 14 or later for PC or 

» Java Runtime Environment (JRE): Java 6.0, Update 10 for Vista on PC to download test 
content ahead of time 

■ Java Runtime Environment (JRE): Java 5.0, Update 7 or later for Mac 

■ Web browser 

• Test Administrator's Workstation: 

IE 6 OR IE 7 (Windows) 
* Firefox 2.0. Note: IE must be used to download test content or install Flash on student 
workstations. 
Safari 2.0 (Mac) 

• Student Workstation: 

■ IE 6 OR IE 7 (Windows) Must be resident on workstation 

■ Safari 2.0 (Mac) 

Each school must provide a sufficient number of computers that meet the minimum requirements in 
order to conduct practice sessions prior to testing. 
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Attachment D 

Single Sign-on 
Spring 2011 and Spring 2012 Work Plan 

The CTB McGraw-Hill online applications are integrated to provide Single Sign-on access for the Indiana 
users. The integrated applications are Navigator™ (TMA and RES), OAS - the online platform for the 
ISTEP+ summative testing, Indiana Online Reporting System (INORS), and Acuity - for online formative 
testing. Users have one set of credentials that are used to access all the integrated applications through a single 
entry point at <www.ctb.com/entrv> . 

The feature is available to corporation and school administrators. The single sign-on functions can be used by 
teachers as well when their accounts are migrated (currently not in scope) after a more consistent use of 
Personnel Identification Number (EIN — external identification number) is achieved. This feature is not 
available for students. 

The key functions available to users with single sign-on access: 

Single user account and single point of access for multiple applications 

Replication of key user attributes (password, email address, phone, address) across multiple systems 
Forgot password/ password reset 

Unique organizations structure (state, corporation, schools) across multiple systems 
Replication of districts/ schools (organizations) information across multiple systems 

Users in corporations and schools testing ISTEP+ on OAS will receive access to OAS approximately 4 weeks 
prior to the testing window start and will have the access removed at the end of the business on the last day of 
testing. 

As new corporations/ schools start testing Acuity, if they were previous ISTEP+ participants, their accounts 
will be maintained and will be provided access to Acuity through single sign-on. 

End user documentation for single sign-on will be available on <www.ctb.com/istep> and will be updated as 
needed. 

Periodical enhancements to the single sign-on system or data migration work will be schedule as much as 
possible outside of business hours or, when down time is required, in agreement with the 1DOE. 
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Attachment E 

Item Development 
Spring 2011 and Spring 2012 Work Plan 



ELA 3-8 / Math 3-8 / Science 4&6 / Soc St 5&7 20% Online 
Operational Only Extension for Spring 2011 Administration 



ISTEP Operational March 201 1 - <cc Dec 2010) - open-ended "Writing" 



1 form per grade 




Form 
Oeslnn 


Od Materials 




ONLINE 


Printed Online Admin 


Materials 


Grade 


Content 


Item 
Type 


# Items per 
form 


Scannable 
Test Book 
- Pgs per 
form 


# forms 


Total Pgs 




Load 
Items 
for 
online 


%of 
students 
online 


Math& 
Science 
Scannable 
CR/ER 
books 


forms 


Total Pgs 






CR 


3 










3 












ELA 


ER 


1 










1 










3 




prompt 


1 


32 


1 


32 




1 


20% 










Math 


CR 


3 














12 


1 


12 




ER 


1 


















CR 


3 










3 












ELA 


ER 


1 










1 














prompt 


1 










1 










4 


Math 


CR 


3 


48 


1 


48 






20% 










ER 


1 














20 


1 


20 




Science 


CR 


3 
















ER 


1 
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3 
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All tests: Large Print & Braille 
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Online Students have a printed book for Math&Sclence only 
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ISTEP Operational Spring 201 1 - (cc Dec 201 0) - MULTIPLE CHOICE "Progress" 



1 form per grade 

Item 

Grade Content Type 


Dosicfn- 


'~ • ~ Mated 




Items per 
form 


Non- 
Scannable 
Test Book 

PflS 


9 forms 


Total Test 

PflS 


ocannsoie 
Answer . 
BookPgs 


forms 


Tntat 
1 Dial 

Answer 

PflS 


3 


ELA 


SR 


•to 


n/a - test book is scannable 




1 




Math 


SR 


Aft 
40 


4 


ELA 


SR 


45 


f'T "v'V'T "*\-"\> 
^112 V 
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8 
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SR 


46 
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SR 


45 


5 
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All tests: Large Print & Braille 
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Form 
Deelnn 


Pilot 1 


otals 


Pilot Materials 


Grade 


Content 


Item 
Type 


# Items per 
form 


# forms 


Total 
Pilot 
Items 
across 
forms 


Scannable 
Test Book 
* Pgs per 
form 


# forms 


Total 
Pfls 


3 


Math 


CR 


5 




5 


ID 




ID 


ER 


2 


1 


2 




4 


Math 


CR 


5 




5 


16 


1 


16 


ER 


2 




2 




5 
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CR 


5 




5 


16 




16 


ER 


2 


1 


2 




6 
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CR 
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5 
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16 


ER 
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7 
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CR 


5 




5 


16 




16 


ER 
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8 
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CR 
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5 


16 




16 


ER 
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42 



96 



ISTEP MC MATH Pilot Spring 2011 (cc 



Dec 2010) 



Item 

Grade Content Type 


Form 
Desitin 


Pilot 1 


totals 


Pilot Materials 


9 Items per 
form 


# forms 


Total 
Pilot 
Items 
across 
forms 


Scannable 
Test Book 
•Pga per 
form 


# forms 


Total 

PflS 


3 
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MC 


46 




46 
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32 


4 
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MC 


46 




46 
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32 


5 
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MC 


46 




46 


32 




32 


6 
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MC 


46 




46 


32 




32 


7 
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MC 


46 




46 


32 




32 


8 
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MC 


46 




46 


32 




32 



276 6 192 
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ELA 3-8 / Math 3-8 / Science 4&6 / Soc St S&7 20% Online 



Snrinn 2011 AncillariAR 

wMI II IM 1 f IVMIflf IUO 




. N0t68 


Punch-Out Manipulates 




reprint / ship per admin window 


Punch-Out Manlpulatives - Large Print 




reprint / ship per admin window 


Manipulates - Braille 




reprint / ship per admin window 


Practice Test MC-gr3 


8 


pdf posted only (same as last year) 


^Pra&c^Test^ 




■' ' 1 ' ."' ' ' pdf posted only (same as last year) 


Practice Test - gr 6-8 


16 


pdf posted only (same as last year) 


ONLINE Practice Test - gr 3 (6 items - 
mixed contents) 




OAS only (same as last year) 


^NtllNE; RractiMiTe^^ 

it mixeaconienis/,, ^j^ff^.'. , ( n , 


iM 


' OAS only (same as last year) 


' ONLINE , Practice Test - gr^o (//Items--! 
j; mixed 'CoritehtsV''', i' l! : Ti"V'N^'k*4hriH | i''' , <" iiiiiiK 


.i? ■;!!.. -fa. 1 . 


ii.,.. 'OAS only (same as last year)' 


; ONI '(NP PractlM^T6atft!.,Gre'fcfiViO'. , '' 


i.i ■■ " ■ 


■ OAS only (sarhe as last year) - . '■ ' 


Admin Manual - gr 3 




print and post online 


Admin Manual - gr4 


104 


print and post online 


Admin Manual - gr 5 




print and post online 


Admin Manual - gr 6 




print and post online 


Admin Manual - gr 7 


■1112 ; < 


print and post online 


Admin Manual - gr 8 




print and post online 


ONLINE USERS MANUAL 




This Is incorporated Into each EM ••> ; 


Test Coordinator Manual 




par posieo oniy 


Packing Poster #1 ("writing" admin) 


1 


ship 1x for "writing" admin 


Packing Poster #2 ("progress" admin) 


1 


ship 1x for "progress" admin 


Guide to Test Interpretation - gr 3-8 


56 


pdf posted only 


Teacher Scoring Guide - grade 3 (all 
contents) 


72 


pdf posted only 


Teacher Scoring Guide - grade 4 (all 
contents) 


88 


pdf posted only 


Teacher Scoring Guide - grade 5 (all 
contents) 


88 


pdf posted only 


Teacher Scoring Guide - grade 6 (all 
contents) 


88 


pdf posted only 


Teacher Scoring Guide - grade 7 (all 
contents) 


88 


pdf posted only 
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Teacher Scoring Guide - grade 6 (all 
contents) 




pdf posted only 


Audio Tapes - gr 3,4,5 













OPTION NOT COSTED AT THIS TIME (Audio Tapes gr 6,7,8) 

ELA 3-8 / Math 3-8 / Science 4&6 / Soc St 5&7 20% Online 
Operational Only Extension for Spring 2012 Administration 



1 form per oracle 




•^orm 
: Desidn 


- 


3d Materials 




Grade 


Content 


Item 

lypB 


# Items per 
form 


Scannable 
Test Book 
- Pgs per 
fonn 


fnrma 

w lunns 








PR 


«a 










FI A 
Cl_/*\ 


ER 


1 








3 




□romot 


1 


32 


1 


32 




Math 


CR 


3 










ER 


1 












CR 


3 


'-..J- ■. .' 








ELA 


ER 


1 












prompt 


1 








4 


Math 


CR 


3 


48 ' 


1 


7. 48. 




ER 


1 










Science 


CR 


3 










ER 


1 






■ ■■- 1 ■ ■" 


5 




CR 


3 


yr"48"-'" 


1 


v;~ : 48vr 




ELA 


ER 


1 












prompt 


1 


'- ! ' 








Math 


CR 


3 










ER 


1 










Social 


CR 


3 









- y ■ — 

l"" ' ONLINE 


- Printed Online Admin Materials. ':. 


Load 

Itoftin 
iKinio - 

for 

online 


%of 
students 
online 


Math& 
Science 
Scannable 
CR/ER 
books 


# forms 


Total Pus 


3 


20% 


■7 r- — — sr — 1 .;™ irr 11'*""* — .T"*'" "r" 

"'■ '1>. \ i" ■ ii'' 1 ' V'f , W*>;vV; ; 

r , V i %■ ii ■ - tr 


1 


1 




12 


1 


12 


f / 


3 


20% 




1 


1 




20 


1 


20 




- --- - -v 




3 


20% 


• " ; " <•**.■>■ 'r- , i-"' 
■-- ,- ( .y.'-" .". - ■ \ qifi 

i, , . .. , ', 'u , . " i 


1 


1 




12 


1 


12 




3 
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Studies 


ER 


1 












CR 


3 










ELA 


ER 


■ 1 






f V '" v: 






prompt 


1 


, ■ ■ 






6 


Math 


CR 


3 


;.. ;: 48:,- ; 


1 






ER 


1 










WWIOI Ivv 


CR 


3 


K •.' '"' ' ; '-r- : , 








ER 


1 












CR 


3 


■} 








ELA 


ER 


1 


'V, ' ' 










prompt 


1 


.- a "■ ■ , 
' " i • " ' 1 ' 




h.'r ■ ' ' •! 
'-r ■- . < ' 


7 


Math 


CR 


3 


48-- < ■ 


1 


•;V48;;v- 




ER 


1 






> * l ■ 




Social 


CR 


3 


1 1 n ~ ■■ . 








Studies 


ER 


1 












CR 


3 






• ') " -; 




ELA 


ER 


1 






•<.*'•-;•*/ 






prompt 


1 


> iV "'" V ■■: 




S-t _ ^ " '.' ■ . 
-s : : * 


8 


Math 


CR 


3 


40 :i 


1 


... 




ER 


1 


v - ;'" " 








'Studies 

■■■".fViliK"""!.' ■ 


;«*!' ^.CKit 


"cr - ; 










,vER"v? 











1 




-' •: ""'^•."y-j".: .r^5.';,-W" -'^ 


3 


20% 




1 


1 




20 


1 


20 


' .' " ^ 


1 , ■■ 




3 


20% 


>*J: •^' ! :. C V^^^'J"' ■„ 


1 


1 


' v 


12 


1 


12 




3 




1 


3 


20% 


^ *" - . , ,■* ,' ■ .*'r v ; i ' . i 

- ml ■ 1 tt. < •■ 
". . ' ') " - ' ? ^ J • ■*. 


1 


1 




12 


1 


12 








.. - .0 . - 



All tests: Large Print & Braille 



Online Students have a printed book for Math&Science only 
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ISTEP Operational Spring 20 



2 - (ccDoc 2011) - MULTIPLE CHOICE "Progress" 



1 form per grade 

Item 

Grade Content Type 


form 
Deslan 


Materials 


# Hems per 
Torro 


Non- 
Scannable 
Test Book 

PBS 


# forms 


Total 
Test Pfls 


QCannaDlf) 

Answer 
Book Pas 


forms 


1 Owl 

Answer 


3 


ELA 


SR 




nVa - test book is scannable 


76 . 


1 




Math 


SR 


Ad 


4 


ELA 


SR 


45 


112 


1 


, 112 


8 


1 


8 


Math 


SR 


46 


Science 


SR 


45 


5 


ELA 


SR 


45 


112 


1 


112 


8 


1 


8 


Math 


SR 


46 


Social 
Studies 


SR 


AC 

45 


6 


ELA 


SR 


45 


an 


A 

1 




8 


1 


■ " ■ 
•12 


Math 
iviau i 


SR 


40 


GR 


6 


Science 


SR 


45 


7 


ELA 


SR 


45 


112 


1 


112 


8 


1 




Math 


SR 


40 


GR 


6 


Social 
Studies 


SR 


45 


8 


ELA 


SR 


45 


■ .96 


1 


96 


8 


1 


12 . ' 

. - .. 1 . ] 


Math 


SR 


40 


GR 


6 


Social 
Studies 


SR 






ONI 


JNE 


Load 
Items for 
online 


%Of 

students 
online 


AC. 


20% 


ACt 
4D 


45 


a An/ 

20% 


46 


45 


45 


20% 


46 


45 


45 


20% 


40 


6 


45 


45 


20% 


40 


6 


45 


45 


20% 


40 


6 






All tests: Urge Print & Braille 
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dated: 03/14/08 



Sorina 2012 Anci Maries 




Notes 


Punch-Out Manipulates 




mnrint / Qhfn nor oHmin uiinHnutr 

1 v^JI 1(11 1 9111^ ^JO( QUI Mill nil lUUn 


Punch-Out Manipulatives - Large Print 




reprint / ship per admin window 


nnanipuiaiivoB dibiiio 




reprint / ship per admin window 


Practice Test - gr 3 


8 


pdf posted only (same as last year) 


Practice Test MC- gr 3 


8 


pdf posted only (same as last year) 


Practice Test AS?'gr, 3 ' ' ..V . . ^ [ \ ■ ■'.,> 


8 , 


pdf posted only (same as last year) 


Practice Test - gr 6-8 


16 


pdf posted only (same as last year) 


ONLINE Practice Test - gr 3 (6 items - 
mixed contents) 




OAS only (same as last year) 


,:6NLINE'iPrac1ic^ 

mtXep Contents); " ;„?-,.' .(j.i'jil; 




OAS only (same as last year) 


.ONLINE Pradlce Test -,gr L 6 (7 items : 
;!mlxea:c™^ 




\jf\o oniy (same as lasi vearj . 


; ONLINE Practice Tests- Grs 6-8 MO - 
iterns -. mixed; contents) . - v.V ■/ 


r-f-r- 

'(l.ViV ... 

"I ' 


-OAS only (same as last year) 


Admin Manual -gr 3 


to'96"i. 


print and post online 


Admin Manual - gr4 


■104 


print and post online 


Admin Manual - gr5 


3-104 '. 


print and post online 


Admin Manual - gr 6 


412 


print and post online 


Admin Manual - gr 7 


,; ii2 ■ 


print and post online 


Admin Manual - gr 8 


-98 


print and post online 


ONLINE USERS MANUAL 


,; 32* : ' : 


' This Is incorporated into each EM 1 


ToqI ^AnrHinotnr Manual 

1 COL uvul UJI ICILUI IYH3I lUIJI 




par posted only 


Packing Poster #1 ("writing" admin) 


1 


ship 1x for "writing" admin 


Packing Poster #2 ("progress" admin) 


1 


ship 1x for "progress" admin 


Guide to Test Interpretation - gr 3-8 


56 


pdf posted only 


Teacher Scoring Guide - grade 3 (all 
contents) 


72 


pdf posted only 


Teacher Scoring Guide - grade 4 (all 
contents) 


88 


pdf posted only 


Teacher Scoring Guide - grade 5 (all 
contents) 


88 


pdf posted only 


Teacher Scoring Guide - grade 6 (all 
contents) 


88 


pdf posted only 
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Teacher Scoring Guide - grade 7 (all 
contents) 


88 


pdf posted only 


Teacher Scoring Guide - grade 8 (all 
contents) 




pdf posted only 


Audio Tapes - gr 3,4,5 
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Attachment F 
Program Milestones and Critical Dates - Spring 2011 



Soring 2011 

The following, with the exception of the test windows, are tentative critical dates for Work Scope 
hereunder. Key Deliverables that are subject to Liquidated Damages are indicated with a dollar 
sign ($). The tentative dates below are based on the original scope of work for the contract 
extension which does not include any adjustments that may be necessary for sell-up options 
chosen by the 1DOE. The parties agree to negotiate all tentative dates during the initial planning 
meetings for that contract year and to update or revise the dates in accordance with Paragraph 8 
"Changes in Work" of the Contract. 



rpr the First Year of the Renewal Period 







Applied Skills Test Materials In-site $ 


0/1/1 1 _9/in/i l 


Applied Skills Test Window 


9/9x71 1 — "VQ/1 1 


Multiple-Choice Test Materials In-site $ 


3/30/11 -4/11/11 


Multiple-Choice Test Window 


4/25/11-5/4/11 


Multiple-Choice Online Test Window 


4/25/11-5/6/11 


INORS Live (Phase 1)$ 


5/26/11 


Parent Network Live (Phase 1 ) $ 


5/26/11 


Parent Network Student Report In-Site (Phase 2) $ 


6/14/11 


Student Report Labels in-site (Phase 2) $ 


6/14/11 


Image Print Live (Phase 2) $ 


6/14/11 


INORS Summaries Live (Phase 3) $ 


6/14/11 


Revised DISAG and GRTs Live (Phase 4) $ 


8/10/11 
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Attachment G 

Program Milestones and Critical Dates - Spring 2012 

Spring 2012 

The following, with the exception of the test windows, are tentative critical dates for Work Scope 
hereunder. Key Deliverables that are subject to Liquidated Damages are indicated with a dollar 
sign ($). The tentative dates below are based on the original scope of work for the contract 
extension which does not include any adjustments that may be necessary for sell-up options 
chosen by the IDOE. The parties agree to negotiate all tentative dates during the initial planning 
meetings for that contract year and to update or revise the dates in accordance with Paragraph 8 
"Changes in Work" of the Contract. 



For the Second Year of the Renewal Period 



Deliverables^, 'x 'jgf. .>_-_' .^C-.v^C ? ^'-i--L-*l-*\ >r.'?/£l- . 7 .V- 




Applied Skills Test Materials In-site $ 


j/ji/iz — uyixi 


Applied Skills Test Window 


3/5/12-3/14/12 


Multiple-Choice Test Materials In-site $ 


3/28/12-4/9/12 


Multiple-Choice Test Window 


4/30/12-5/9/12 


Multiple-Choice Online Test Window 


4/30/12-5/11/12 ' 


INORS Live (Phase 1)$ 


6/01/12 


Parent Network Live (Phase 1) $ 


6/01/12 


Parent Network Student Report In -Site (Phase 2) $ 


6/19/12 


Student Report Labels in-site (Phase 2) $ 


6/19/12 


Image Print Live (Phase 2) $ 


6/19/12 


INORS Summaries Live (Phase 3) $ 


6/19/12 


Revised DISAG and GRTs Live (Phase 4) $ 


8/15/12 
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ISTEP+ Contract - Program Enhancements EXHIBIT B 

V1.4 



DCF/YEARfS) 


METRICS 


PRICE 


PM06-S11/S12 

Secure Barcodes 
and Inventory 


Assumptions: 

Secure Barcode and Inventory isn't included in the contract extension it was a "sell up" for 2009 and 2010. 

• Secure Barcodes for 80% paper and pencil test books only 

• Inventory costs to be included 

• Please separate the costs for years S 1 1 and S 1 2 

• No Pilot books included in costs 

• Include secure barcodes and inventory for LP and Braille in grades 3-8 

• Include secure barcodes and inventory for audio tapes grades 3-5 

Indiana Model for Inventory (ScanOptics): 

1. Secure materials bar-coded at printer, BC files to CTB 

2. When returning materials, customer separates material by secure (shipped to either Scoring Center . 
IN or ScanOptics CT) and non-secure (eventually destroyed by customer) 

3. Graphical packaging instructions (TCM & Poster) 

4. Dedicated carrier/secure pickup . , . . 

5. Customer called by either site if discrepancies in box counts 

6. Immediate scan of all used and unused materials . 

7. Corporations with missing materials are reviewed, re-scanned 

8. Inventory/Missing Materials Report run by Scoring and sent to the IDOE for distribution 

9. Customer audit possible by Scoring if disagreement as to missing materials 

Assumptions: 

1 . Grade 3 for AS and M-C only unused will be sent to ScanOptics. Used will be scanned in 
Indianapolis as they are scored. 

2. Grades 4-8 AS only unused will be sent to ScanOptics. Used will be scanned in Indianapolis as 
they are scored. 

3. Grades 4-8 use answer booklets for M-C so all grades 4-8 test books will be sent to ScanOptics. 


$1,345,774 
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Number of Scannable Books that Need Barcodes: 



See Page 1 



Grade 


Print 
Quantity 
SI 1/AS 


Print 
Quantity 
Sll/MC 


Print 
, Quantity 
S12/AS 


Print 
Quantity 
. S12/MC ■ 












3 


97 


76 


97 


76 


4 


97 


76 . 


97 


76 


5 


• 99 


78 


99 


78 


6 


99 


78 


99 


78 


• ... 7 


99 


78 


99 


78 


8 


102 


81 


102 


81 . 
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DCF/YEAR(S) 


METRICS 


PRICE 


PM08-S11/S12 

Test Coordinator's 
Manual Print 


Assumptions: 

The IDOE requested a price on printing the Test Coordinator Manuals' (TCM's) for Spring 2010 and 201 1. 

• Print, package, fulfillment, and shipping of 7K TCM's 

• Manuals will be shipped to corps, packaged by school. 

• Excess used for short/adds during the Multiple-Choice administration 


$18,750 
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DCF/YEAR(S) 


METRICS 


PRICE 


PM09-SU/S12 
Practice Test Print 


Assumptions: 

I DOE has requested costs to print Grade 3 Applied Skills and Multiple-Choice Practice Tests for Spring 
201 1 and 2012 and print Gridded Response Practice Test for Grade 6 Multiple-Choice Spring 201 1 and 
2012. 

• ■ Print, package, fulfillment, and shipping of 78K grade 3 8-page Practice Test for Applied Skills 

Assessment (March) 

• Print, package, fulfillment, and shipping of 78K grade 3 8-page Practice Test for Multiple-Choice 
Assessment (April) 

• Print, package, fulfillment, and shipping of 77K grade 6 16-page Practice Test for Gridded 
Response (April) 


$105,988 
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DCF/YEARfS) 


METRICS 


PRICE 


PM10-S11 
SIQ Changes 


Assumptions: 

This change will be for contract year 2010-1 1 (spring 201 1 administration) and forward 

Based on initial review, the following functional areas are impacted with Ethnicity changes: 

a. Development (changes to the bio/SIQ pages, TEM, Scoring Guides, (ancillaries) etc.) 

b. Technology (capture and use new rules for resolving ethnicity for students, pre-id, RES, DISA 
report changes, any customization to OAS as applicable) 

c. GROW (INORS DISA report, new rules, compatibility in reporting new and old rules - prior year 
reports will follow old data rules) 

d. Research (use new rules for validation school selection, sampling, etc.) 

The Teacher code change will impact the following functional area: 

a. INORS (use of teacher codes to create accounts, linking of student to teachers) 

b. DEx (rules for incorporating the teacher codes and associated users into the system as opposed to 
what is currently in the design) 

c. No change to Pre-ld, RES as existing (2010) rules will be continued to be followed by 
Technology systems. Any new features will be out of scope and will be treated as new change 
when known. 

d. No impact to Development & Research as this data is just a pass-through from Pre-Id/RES to 
GRT /INORS. 

Newly Collected Ethnicity/race Info 
Answer Document 

• Will collect the information according to the new federal guidelines 

o Hispanic/Not Hispanic (mark one) 
o Race - 5 categories (mark all that apply) 
Pre-IDFHe- 

• Will provide the collected information (6 buckets) 

o Hispanic - yes/no 

o Race Asian - yes/no 

o Race Black/Non-Hispanic - yes/no 

o Race American Indian - yes/no 

o Race Native Hawaiian - yes/no 

o Race White/Non-Hispanic - yes/no 

OAS 

• The display of race/ethnicity info will change to resemble the answer document. 


$41,818 
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PM10-S11 (Cont) 


DmhIvpH Fthnirltv/R a4*p Pnfwinrv 


See Page 5 




* cuiniciiy/i\acc win oe resoivea 10 one 01 uie iouowing repuning categories using a sei or ruies 




SIO rhnnora 








n Asian 






n Rlactc/Nnn-Wisnanie 






o American Indian 






o Native Hawaiian 






o , White/ Non-Hispanic 






o Two or More Races 






* Assumes resolution takes place in bridge 






RES 






• Show the resolved/reoorted cateeorv in RES fno chanced 






o If the school/corp. updates the resolved/reported category, it will be out of sync with the 






collected information. We assume this is OK. 






Summary 






• Assumes a change to the DISA records 






Reports 






• Assumes a change to the Disaggregation Summary File to report the 7 categories plus the No 






Valid Information category. 






• GRT - assumes we add the collected Ethnicity/Race information (6 buckets). 






o Continue to report the resolved/reported category 






INORS 






• Changes to the Disaggregation Summary Report 






• GRT will have a new record length 






Parent Network 






• Load a GRT with a different record length 
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DCF/YEARfS) 


METRICS 


PRICE 


PM11-S11/S12 


Assumptions: 








$1,305,876 


100% Paper and 
Pencil 

- 


Multiple-Choice 100% paper due to less than 20% online testing 
Manufacturing 

• ~2 1 K Multiple-Choice test books at grades 3 and -2 1 K test Multiple-Choice test books 
and -2 1 K answer books at grades 4-8 at each grade 

• Practice Tests -2 1 K at grade 3 for A-S and -2 1 K for M-C and -21K at grade 6 for M-C 
Gridded Response (separate price) 

• Secure Barcodes/Inventory for 20% materials 

Fulfillment: 

• Packages of 6 and 12 for 126K test books grades 3-8 and 105K answer books grades 4-8 
. • . Packages of 6 and 12 for grade 3 and grade 6 Practice Tests (separate price) 

• Costs for materials shipped to p/p vendor if applicable 

ShlDDlns: 

• Delivery of materials to corps/schools 

• Pickup at corps/schools for shipment to Indianapolis & ScanOptics) 
Scorine Operations: 

• 20% increase in cases/scan-sheets and inventory 








Year 1 


2011 


Practice Tests 










Grade 


Books 


Administration 










3 (8-pager) 


21K 


AS.(March) 










3 (8-pager) 


21K 


M-C (April) 










6 (16-pager) 


21K 


M-C (April) 
























Year 2 


2012 


Practice Tests 










' Grade 


Books 


Administration 










3 (8-pager) 


21K 


AS (March) 










3 (8-pager) 


21K 


M-C (April) 










6 (16-pager) 


21K 


M-C (April) 
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DCF/YEARfS) 


METRICS 


PRICE 


PM12-S11/S12 

OAS Multiple- 
Choice Virtual 
Form 2 


Assumptions: 

To create a virtual form for OAS purposes for Multiple-Choice by re-arranging the existing items using the 
current year Operational form (not any other year). The item re-arranging needs to be limited within a 
subtest (distracters will not be re-arranged). The purpose is to minimize the risk of student copying or 
cheating when testing online. There will be no change to the paper test form. 

The following are potential areas of impact: 

• PEID 

• OAS content data load 

• Technology Back End systems 

• Possible impact to manual clean up 

Assumptions - Technology 

• The alternate form will be processed as a new Multiple-Choice form through the Back End 

• There will be no field test items 

. ■• OAS will set the alternate form before export to the mainframe 

• The Front End will not process the alternate form or load the PEID 

• A new PEID will be loaded to the Back End. The reporting structure (content area/objective) and 
titles will be the same as the regular form 

• If items are suppressed they will have to be suppressed in the (regular paper/pencil and OAS 
form) and the alternate OAS form. 

• Bridge/CR checker/match merge/unmerge will be modified to include the new alternate form. 

• The alternate form will use the same tables and cuts as the regular form 

• The alternate form will be reported along with the regular form 


$89,724 
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DCF/YEARfS) 


METRICS 


PRICE 


PM16-S11 

Delete INORS 
Report(s) 


Assumptions: 

Technology Project Management costs 


$0 
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DCF/YEARfSV 


METRICS 


PRICE 


PM19-S11 

De-Scope SI 1 
Applied Skills OAS 
Testing . 


Assumptions: 

Based on the feedback received and observing student experiences in typing CR responses during OAS 
Fall Pilot test administration in 2008, the I DOE has made the decision not to administer the Applied 
Skills OAS Test administration in S 1 1 . 

The IDOE plans to administer in subsequent years (from Spring 2012 onwards).- The IDOE recommends 
including typing skills as one of the skills to be added to their curriculum. 

• Content/Publishing does not have to edit and publish this test in OAS. 

• OAS-Technology to not administer the test (no development, QA, or implementation); update 
the schedule/milestones for dealing with Technology Coordinators and Test Coordinators 
training plans for the OAS Multiple-Choice test administration. 

• EISS Technology to not implement Winbridge/Match-Merge/handscore upload functionality 
for OAS student responses; no need for image XML file handoff. 

• Technology will not have to deal with additional image XML file for OAS student responses 
and delivery of the same in Parent Network and INORS. 

Removal of the following activities would apply: : 

• Changes to main Bridge program 

• Changes to the Bridge User Exit for project customization 

• Changes to the ESOP Services Screens for match/merge system 

• Applied Skills site readiness - 20 sites (note that the maximum # of sites per year was capped at 
45 total for both Applied skills and Multiple-Choice 

• Extracting the data after the Applied Skills test administration 

• Support during 5 days of practice and 1 days of operational testing 

• Support developing custom documentation 

• Training specific to the Applied Skills part of the Project Management oversight 


($71,978) 
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DCF/YEARfS) 



PM21-S11/S12 
GQE Storage 



METRICS 



PRICE 



Assumptions: 

• 2011 : 2007 GQE materials stored - $15,214 

• 2012 : 2008 GQE materials stored - $2,357 ' 



$17,571 





Current 




Current 






Previous 






2007 


: . 6b; . r 




2008 


;o*-TivV 




Total Cages 


■"'71'^ 
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DCF/YEAR(S) 



METRICS 



PRICE 



PM22-S11/S12 
GQE Destruction 



Assumptions: 



2011 : 2007 GQE materials to be destroyed - $6,857 
2012 : 2008 GQE materia Is. to be destroyed - $1,143 



fr'Wka^M';' '- ' i- >' : ^^^'-^..IndianaWiifehousctf of cages)' 



$8,000 



Current Previous 


Current 




2007 






2008 






Total Cages 
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nCF/YFARfS"* 


METRICS 


PRICE 


PM36-S11 


Assumptions: 




$45,035 


Science 


• CTB/IDOE: Conduct Science Specification Meeting (January 201 1) 




Specification 


o 


2-day Specifications Meeting to accomplish the following: 




Meeting 




a. Determine/confirm appropriate item types for new standards/indicators 

b. Make additional revisions to blueprint as needed 

d. Align item poo! to new Science standards to determine exact item 
. development needed 









10 Attendees per day 






o 


15 Overnight attendees 






o 


1 meeting room per day plus (total of 2 = 4 over 2 days) 






o 


1 AE and 1 CDL to attend from publishing (2-day meeting, prep work prior to 
meeting, 1 day travel at each end of the meeting) 






o 


2 PM attendees 


• 






WDepartmentU^ 














Hotel (20 attendees and PM) 












Meeting Rooms (4) 












Meeting Equipment 












Meeting Supplies 




- 






mg&sm 1 ', ;<• 

ifiManagement/-,-. 


Parking 










Food & Beverage 










Mileage Reimbursement 










Dinner Reimbursement 












Substitute Pay 












PM Travel 












PM Car/Parking 












PM Meals 












Misc. 
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DCF/YEARfS* 


METRICS 


PRICE 


PM37-S11 

Vl/«l on np I torn 

Development 


Assumptions: 

Item Development for new Science Standards: 

• Total of 90 MC items (45 items x 2 grades) 

• Total of40CRs (20 items x 2 grades) 

• Total of 12 ERs (6 items x 2 grades) 


$57,998 
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DCF/YEARfS) 


METRICS 


PRICE 


PM38-S1I 

Science Content 
and Bias Meeting 


Assumptions: 

CTB/IDOE: Conduct Science Content and Bias Review Meeting (July 201 1) 

• 2-day Content and Bias Meeting 

• 4 meeting rooms per day (total of 8) 

• 20 Attendees per day 

• Bias/Sensitivity (5-6 teachers) 

K Citizen's Committee (5-6 teachers) 

• 1 AE and 1 CDL to attend from CTB publishing 

Total of 90 MC items (45 items x 2 grades) 
Total of 40 CRs (20 items x 2 grades) 
Total of 12 ERs (6 items x 2 grades) 
Handscoring: 

• 1 LIB Handscoring person per room — 1 people total. 

• Each Handscoring person needed for 2 days 

• Each Handscoring person needed for 40. 


$76,206 



Exhibit B,Page 16 of 19 



DCF/YEARfS) 


METRICS 


PRICE 


PM39-S12 

Science Pilot Test 
and Associated 
Tasks 


Assumptions: 

• Development and production of stand 
alone Science Pilot forms 

• j M-C Pilot per grade (4 and 6) 

• 4 A.S. Pilots per grade (4 and 6) 

• Technology Project Management 

• Scoring Project Management 

• Scaling & Equating 

• Document Processing 

• Items will be scored as a 201 1 pilot 

• 20 Science CR & 6 Science ER items per grade to 
score 

• All training/scoring will be done on day shift 

• Pilot Anchor Rangefinding 

o With teachers 

• 3000 per item scored plus 5% second reads 
■ • Each item will be scored using the current 

analytic rubric for Science CR & ER items 

• 12 day training/scoring window 

• TMRM will not be done with IDOE. IDOE will do independent review of 
TMRM CD's. 

• Update item pool 

• PEID release 

• Operational Form selection for Spring 20 13 


$528,936 
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DCF/YEAR(S) 



METRICS 



PRICE 



PM39-S12(Cont) 

Science Pilot Test 
and Associated 
Tasks 



Publishing 

• 1 Form of LP per grade (4 and 6) 

• No Braille 

ISTEP CR/ER Pilot Spring 2011 



Gradfiffil 


Content Area 


Item 
Type 


Field 
Test 
Items 
per 
Form 


# Forms 

WW I VllllV 


Total 
Field 
Test 
Items 


Total 
Items 
per 
Form 


Total 
Pages 
per 
Form 


«*■#••. 


.'.Science 


MG - 


:' 45 




>■ „45 ■ 


; \<.45 ;,- 


, . -16, 




Science 


CR ; 


: 5-:.' 


' * , '., ,fl ■ 

■•■>, . Ah •<*>..■' 






mm, 




• Science ^ ' 


ER 


,.'2 




"''„,"■ tew'"'''. 1 






■ e • • . 


• Science. ■ 


MG" 


■ -46* • 


V' ■" ■*v * 


' .45 v . 


.;.'.45 


: Me, 




.^Science % - , 


CR 






, V 20 ; 








v Science 


ER • 


- 2 ■ 




:6.;/' 


■ „.< .i2 I... i* 










10 




104 





See Page 16 
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DCF/YEARfS) 


METRICS 


PRICE 


PM40-S11 


metrics 


$0 




In an effort to create a universal Pre-id data layout for all vendors they deal with the IDOE is intending 


Student Barcode 


to modify the layout of the Pre-id file for spring 201 1 . 




Layout Changes 






Pre-id layout changes are acceptable with no additional cost as long as the following hold true: 

a) All data points that are currently used will continue to exist in the same data format (ex. 
numeric, alpha, etc.) 

b) No changes are requested on the pre-id Barcode Label print. 

c) No changes are requested on the Pre-id Roster Report print. 

d) The business rules for the transformation of any data values remain the same as in previous 
years (with the exception of Ethnicity). 
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EXHIBIT C 

CTB/McGraw-Hill 
ISTEP+ 

PRICING AND PAYMENT SCHEDULE 



The following tables indicate the pricing for each year and the payment schedule CTB will 
invoice in accordance with this schedule. 



For July 1.2010- June 30. 2011 



September 30, 2010 


$ 900,000 


October 30, 2010 


$ 1,100,000 


November 30, 2010 


$2,200,000 


December 30, 2010 


$2,804,187 


February 28, 201 1 


$4,685,035 


April 30, 2011 


$ 6,242,662 


June 30, 2011 


$1,730,978 


Total 


$19,662,862 


For Jnlv 1. 2011 - Jane 30. 2012 




July 31, 2011 


$ 4,950,000 


September 30, 201 1 


$ 976,206 


October 30, 2011 


$ 1,100,000 


November 30, 2011 


$ 2,200,000 


December 30, 2011 


$ 2,762,369 


February 28, 2012 


$4,640,000 


April 30, 2012 


$ 6,152,938 


June 30, 2012 


$ 7,205,323 


Total 


$29,986,836 
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Exhibit D 

RFP7-93 
SECTION ONE 

GENERAL INFORMATION AND REQUESTED PRODUCTS/SERVICES 

1.1 INTRODUCTION 

In accordance with Indiana statute, including IC 5-22-9, the Indiana Department of 
Administration (IDOA), acting on behalf of the Indiana State Board of Education (the 
State Board) and the Indiana Department of Education (IDOE), requests proposals on all 
or one or more of the components of a long-term statewide student learning assessment 
system for use in the state's K-12 schools (the Assessment System). The content of this 
Request for Proposals (RFP) has been prepared by, or is based upon input from, staff of 
the IDOE and members of the State Board. The overall approach of the Assessment 
System upon which this RFP is based is discussed in "A Long-Term Assessment Plan for 
Indiana: Driving Student Learning," included here as Attachment A. Deliverables for the 
components of the system are discussed below. 

It is the intent of IDOA to solicit responses to this RFP in accordance with the statement 
of work, requirements for preparation of a proposal, and specifications contained in this 
document. This RFP is being posted to the IDOA website (http://www.IN.gov/idoa/proc) 
for downloading. A nominal fee will be charged for providing hard copies. Neither this 
RFP nor any response (proposal) submitted hereto is to be construed as a legal offer. 

1.2 DEFINITIONS AND ABBREVIATIONS 

Following are explanations of terms and abbreviations appearing throughout this RFP. 
Other special terms may be used in the RFP, but they are more localized and defined 
where they appear, rather than in the following list 

Academic Standard A broad content statement of what an individual should know and 
be able to do relative to a particular area of study, (e.g., 
Computation) 

Academic Indicator A subdivision of an Academic Standard that defines specific skills 
and knowledge an individual should learn, (e.g., Understand the 
special properties of and 1 in multiplication and division.) 

Analytical Services Services provided by the contractor to ensure the technical 

soundness of assessments. Such services may include, but are not 
limited to, the selection of representative samples of Indiana 
students, conducting pilot studies, scaling of items, item bias 
analyses, setting of performance standards, and demonstration of 
test reliability and validity. 

Applied Skills Constructed response assessment items 
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Board 



Indiana State Board of Education 



Calibration Placing items on a common scale so that items measuring the same 

underlying content represent a position on the scale that is also 
represented by other items of comparable difficulty. 

Constructed 

Response A test question or task that requires the examinee to create his or 

her own response (e.g., short answer, essay). 



Criterion-Referenced 

Assessment An assessment referenced to pre-defined content standards and 

designed to measure student achievement relative to those 
standards. As used in this RFP, the term means an assessment 
designed to measure achievement relative to the Indiana Academic 
Standards in English/language arts, mathematics, science, and 
social studies. 

Diagnostic or Formative 

Assessment A process used by teachers and students during instruction that 

provides feedback to adjust ongoing teaching and learning to 
improve students 1 achievement of intended instructional outcomes 

Full Time Equivalent A measurement of an employee's 

(FTE) productivity on a specific project or contract. An FTE of 1 would 

mean that there is one worker fully engaged on a project. If there 
are two employees each spending 1/2 of their working time on a 
project, that would also equal 1 FTE. 



GQE Graduation Qualifying Exam; administered in the fall to grade 10 

students, and in both the fall and the spring to students who have 
not yet achieved a passing score. 

IAC The Indiana Administrative Code. 

IC * The Indiana Code. 

IDOA Indiana Department of Administration 

IDOE Indiana Department of Education 

Interim or Benchmark 
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Assessment 



ISBE 



An assessment designed to measure the progress of one or more 
students in a class, a grade, a school, or a corporation in mastering 
the Indiana Academic Standards. , 

Indiana State Board of Education 



ISTEP+ 



Logistic Services 



Mixed Format 
Assessment 



Offeror 
Online 

Other Governmental 
Body 



Passing Score 

Products 
Proposal 



Indiana Statewide Testing for Educational Progress-Plus, Indiana's 
current statewide testing program, currently including statewide 
assessments in English/language arts, mathematics, and science. 
ISTEP+ assessments are currently administered in the fall of the 
year at grades 3 through 10 in English/language arts and 
mathematics. ISTEP+ science currently is administered in the fall 
in grades 5 and 7. 1 

The timely packaging and shipping of test materials and 
administration manuals to all local school corporations in Indiana, 
as well as the timely return of test results to IDOE and local 
corporations and schools. 



An assessment utilizing a variety of types of assessment items, 
including selected response items and constructed response items. 

Respondent to this RFP. 

Communication of information via the internet (World Wide Web) 



An agency, a board, a branch, a bureau, a commission, a council, a 
department, an institution, an office, or another establishment of 
any of the following: 

(1) The judicial branch. 

(2) The legislative branch. 

(3) A political subdivision (includes towns, cities, local 
governments, etc.) 

(4) A state educational institution 

Performance standard adopted by the Indiana State Board of 
Education representing the skills and knowledge that all students 
should acquire as assessed. 

Tangible goods or manufactured items as specified in this RFP. 
An offer as defined in IC 5-22-2-17. 
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Respondent 



RFP 



An offeror as defined in IC 5-22-2-18. TTie State will not consider 
a proposal responsive if two or more offerors submit a joint or 
combined proposal unless one offeror is clearly identified as the 
Respondent who will be ultimately responsible for performance of 
the contract 

Request for Proposals 



School Corporation Indiana's term for a school district or local education agency. 



Scoring Rubric 

Selected Response 

Services 
State 

State Agency 



STN 
Vendor 

Web-Based 



A set of standard rules and procedures used to assign scores to 
students' responses to constructed response items or performance 
assessment tasks. 

A test question that requires the examinee to choose the 
correct/best answer from a list of possible responses (i.e., multiple 
choice). 

Work to be performed as specified in this RFP. 
The State of Indiana 

As defined in IC 4-13-1, "state agency" means ah authority, board, 
branch, commission, committee, department, division, or other 
instrumentality of the executive, including the administrative, 
department of state government. 

Student Test Number, a unique identifier for each student tested. 

Any successful Respondent selected as a result of the procurement 
process to deliver the products or services requested by this RFP. 

Delivered via the Internet World Wide Web. 

13 PURPOSE OF THE RFP 



The purpose of this RFP is to select one or more Respondent(s) that can satisfy one or 
more of the components for the Assessment System. It is the intent of the IDOE to 
contract with one or more Respondents(s) to provide the components for the Assessment 
System. 
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1.4 SUMMARY SCOPE OF WORK 



Overview 

Indiana currently has 293 public school corporations and 45 charter schools, as well as 
295 private schools whose students also participate in the state's assessment system. All 
of these schools serve some 1,11 7,600 students in grades Kindergarten through 1 2, 
including approximately 70,000 students with disabilities and 14,000 students for whom 
English is their second language who require accommodation on the current assessments. 

During the 2006-2007 school year, the State has had the following numbers of students in 
grades Kindergarten through 12: 



Grades 


2006-07 Public School Students 


K 


79,118 


1 


82,623 


2 


79,264 


3 


77,983 


4 


78,135 


5 


79,647 


6 


79,908 


7 


80,934 


8 


82,013 


9 


86,754 


10 


. 82393 


11 


76,653 


12 


69,574 



In 2006, the Indiana Legislature passed HEA 1240 instructing the State Board and the 
IDOE to reexamine the state's K-12 student assessment system. HEA 1240 assigned to 
the State Board and the IDOE the responsibility to develop a long-term plan for student 
assessments. The Board was instructed to review the current assessments in grades 3 
through 10 and develop a system that: 

■ Reflects a student's proficiency in and mastery of the state's academic standards; 

■ Is, to the greatest extent possible, more concise, less time consuming, and less 
expensive to administer than the current tests while maintaining the current level of 
rigor of the tests; 

■ Provides prompt results to students, parents, and teachers; 

■ Explores all options for timing and use of summative tests, including giving a 
summative test in the fall or the spring; 

■ Measures individual student growth from school year to school year, 

■ Explores all options for diagnostic tests for use by teachers to support ongoing 
remediation; 

■ Involves a transition to the use of online testing; 

■ Assesses student proficiency in written communication in an effective manner; and 
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■ Moves to the use of online assessments for Core 40 subjects. 

HEA 1240 also repealed IC 20-32-5-20, which had stated: 
"See. 20 Duties of contractor scoring tests 

(a) Hie contractor that the department engages for scoring tests shall identify 
locations in Indiana that provide a supply of labor and other resources necessary to 
provide scoring services for the program. 

(b) The contractor shall relocate to Indiana the contractor's facilities for scoring the 
applied skills parts of tests given under section 6 of this chapter." . 

To develop the long term student assessment plan and program per HEA 1240, the Board 
solicited information from educators, administrators, parents, and the public concerning 
the program; looked at tests and testing practices in use by or in development by other 
states; and solicited information from testing companies concerning parameters and costs 
of tests, steps to be taken to ensure the validity and reliability of the tests, steps to move 
the longitudinal data from the current testing program to the new testing program, and 
other information the State Board and the IDOE considered useful in developing the 
testing program. 

The State Board and the IDOE moved to develop a system that will align assessment to 
student learning and mastery of basic curricular aims; simplify for teachers, parents and 
communities what Indiana wants our students to master, clarify for them how we will 
assess the curricular aims that add up to that mastery; provide assessment results useful in 
determining student learning and progress toward that mastery; and otherwise meet 
legally prescribed parameters. 

The State Board and the IDOE heard from vendors, who responded to a request for 
information by identifying their existing and developing products and services; from 
schools and practitioners, who voiced concerns and suggested ideas; and from the public 
at large, who expressed their views on a variety of issues related to assessment and 
learning. 

On November 1, 2006, the Board adopted the plan attached as Attachment A. Potential 
Respondents are encouraged to review the plan carefully, as it will inform the State's 
consideration of proposals under this RFP. 

Hie Assessment System described in the plan is intended to achieve multiple goals: meet 
the state's obligations under federal and state law concerning assessment of student 
learning; provide the means for each student's learning and preparation to be confirmed 
during the course of the student's K-12 years; and drive student learning by simplifying 
the annual assessments and shifting resources to educators in the schools to use 
assessments to drive learning. Concerning this last element, the strong consensus of the 
input and resulting decision of the State Board was that, while it is obviously necessary to 
meet the State's legal obligations and maintain validity, reliability and other aspects of an 
effective assessment system, those needs must be met by a system that provides more 
direct support to student learning and the instruction that drives student learning. 
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Certain principles articulated in the plan for the assessment system flow through to the 
RFP: 

■ Standards driven, student centered, learning focused. Its curricular aims should 
prepare the student for post-secondary success. It should support learning-based and 
data-driven instruction; performance evaluation and improvement; and accountability 
for educators, schools and school corporations. Its use should include a stronger focus 
on curricular aims (power standards) and a lesser focus on mere memorization; it is 
intended to be simpler and more rigorous. It should serve all students, measuring 
student performance across the full scale of proficiency and mastery, so that it is 
relevant for the excelling, achieving, and struggling student alike. 

■ Support instruction. The system should be based on curricular aims that succinctly 
and clearly describe the desired student mastery of a manageable number of curricular 
aims and the knowledge content that build directly to post-secondary success. It 
should enable teachers to understand what they are to teach. It should provide results 
useful to educators that transfer with the student across grades and between schools 
and between corporations. It should enable the DDOE or others who provide external 
assistance to access results and provide recommendations to improve learning. 

■ Reliable, valid, consistent with assessment standards, and compliant. It should 
measure student mastery of and proficiency in Indiana's standards; meet requirements 
of federal and state law; enable continued use of data from previous ISTEP+ 
assessments; and enable use of historical data for the student, school and corporation. 

■ Effective and efficient. It should maintain rigor and integrity; be less intrusive in 
classrooms; be more integrated with classroom practices; produce results that are 
returned in a timely manner, enable assessments to be submitted and results to be 
returned electronically (and, over time, online); and it should avoid proliferation of 
assessments for students and teachers. 

Components 

Hie plan calls for, and this RFP solicits proposals for all or components of, a statewide 
assessment system that encompasses diagnostic assessments, end-of-year summative 
assessments, end-of-course assessments, and assessment of student preparation for post- 
secondary success. The RFP contemplates that these components may be contracted for 
separately, or in combinations, at the discretion of the State Board and the IDOE. All 
requirements - mandatory requirements, assessment quality, cost, etc. - will be 
considered for each component (and, where applicable, sub-component). 

Consistent with and as explained in more detail in Attachment A, the purpose of this RFP 
is to solicit one or more vendor(s) to create and implement the Assessment System, the 
components of which include the following: 
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1. Grades K-2: In-year diagnostic assessments (and related professional development 
support) that focus on the student's ability to read, comprehend and use numbers, 
available during the school year to classroom teachers in reading and comprehension 
and use of numbers in grades K-2. 

2. Grades 3-8: In-year diagnostic assessments (and related professional development 
support), available for use by teachers in classrooms during the school year to drive 
learning in English/language arts, mathematics, science, and social studies/history in 
grades 3 through 8. These diagnostic assessments will be state-provided but not 
required. They will be provided through the IDOE and will be optional for schools 
and local school corporations to adopt and use at no cost to them. Schools and 
corporations may choose to use assessments other than those provided by the state, 
but costs for such assessments will be borne entirely by the local school or 
corporation. 

3. Grades 3-8: End-of-year summative assessments, to be given statewide toward the 
end of the school year to measure student performance and growth, consisting of both 
a writing assessment and a progress assessment, administered to students in 
English/language arts, mathematics, science and social studies/history in grades 3 
through 8.. These will include: 

a. Annual writing assessments (both responses and essays), given in the 
second half of each school year. 

b. Annual progress assessments that measure key auricular aims, cognitive 
skills and subject area knowledge, given toward the end of each school year. 

Both of these will be based on auricular aims and content expectations from 
Indiana's Standards. Their scores will be used for accountability purposes. 

4. High School: End-of-course summative assessments will be administered for Core 
40 courses in high school, including specifically Algebra I, English 10 and Biology I, 
and extending over time to the basic Core 40 courses. 

a. The end-of-course summative assessments for English 10, Algebra I and 
Biology I will be used for NCLB compliance purposes. English 10 and 
Algebra I will replace the GQE, such that a student who has passed these 
assessments need take no further state-administered summative assessments to 
meet that requirement to graduate from high school. 

b. Other Core 40 end-of-course summative assessments will be used for 
confirmation of performance in Core 40 courses and guidance counseling will 
be available for use in placement in higher education. 

5. High School: An assessment to confirm student preparation for success in post- 
secondary learning and work will be offered to students in the fall of grades 1 1 and 
12. The assessment will be relevant to determining preparation for both post- 
secondary study and post-secondary work. 

6. All Grades: Each student's assessment results will be tracked into his or her learning 
portfolio, which over time will also include other evidence of the student's learning 
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and development progress. The portfolio will track to the student's STN for 
portability vertically through the grades and horizontally across schools and 
corporations. 

The State is interested in melding the individual system components into an overall 
assessment system that both drives and measures each student's annual academic 
progress and overall preparation for post-secondary success. 
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STUDENT LEARNING PORTFOLIO 



tr 

/ r *\ 



Grades 11-12 



POST-SECONDARY PREPARATION ASSESSMENT 



Grades 9-12 
CORE 40 END-OF-COURSE ASSESSMENTS 
[ALGEBRA I AND ENGLISH 1 - REPLACE THE GQE] 
[ALGEBRA L ENGLISH 10, AND BIOLOGY I - MEET NCLB REQUIREMENTS] 
[MATHEMATICS, ENGLISH, SCIENCE, AND SOCIAL STUDIES - SUPPORT CORE 40 

DIPLOMAS] 



Studrat Ccimselme and Carter 



Grades 3-8 
END-OF- YEAR SUMMATTVE ASSESSMENT 
[ENGLISH, MATH, SCIENCE, and SOCIAL STUDIES] 

' -jr " --I 

| 2ND SEMESTER WRTITNG ASSESSMENT >| END-OF-YEAR PROGRESS ASSESSMENT ■ 

i u : 



Grades 3-8 
Vchintary In-Year Diagnoitk Assessments 
[English, Mathematics, Sciences, and Social Studies] 



Grades K 2 

School bmTding brd confirmation of student reading level at the end of second grade 
Vofcmtary In-Year Diagnostic Assenmenb 
[Reading, Use of Numbers, and Comprehension] 



As a complete assessment system, the combination of these components will provide 
vertical articulation through a student's entire K-12 experience, across grade levels, 
enabling teachers, parents, schools, corporations and communities to anticipate, 
determine and react to student learning as it occurs, measured against what it needs to be. 
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The multiple components in the proposed system and the State's desire to identify 
compatible components within the existing and developing assessment practices of 
interested vendors lead to a matrix approach to this RFP, in which vendors may elect to 
bid on one, multiple or all of the components. The State will not consider a proposal 
responsive if two or more offerors submit a joint or combined proposal, unless one 
offeror is clearly identified as the Respondent who will be ultimately responsible for 
performance of the contract In response to proposals received, the State may elect to 
consider proposals in the configuration submitted or to discuss and/or negotiate with 
vendors to reconfigure one, several or all of the components. 

Hie components may be phased in and prioritized consistently with fiscal responsibility, 
logistical, developmental and legal parameters applicable to individual components, and 
the assessment system as a whole. Hie individual components are described in further 
detail below. 

1.5 RFP OUTLINE 

Hie outline of this RFP document is described below: 



s Section) :. , .tv 


• _ - . , ' •' -■ ■ ;:■ • .-" Description . •= : „ " \- : . ^\ ■ ' ..: ? . 


Section 1 - General Information and 
Requested Products or Services 


This section provides an overview of the RFP, 
general timelines for the process, and a summary of 
the products/services being solicited by the State 
Board and the IDOE via this RFP 


Section 2 - Proposal Preparation 
Instruction 


This section provides instructions on the format and 
content of the RFP, including a Letter of Transmittal, 
Business Proposal, Technical Proposal, and Cost 
Proposal 


Section 3 - Proposal Evaluation 
Criteria 


This section discusses the criteria to be used to 
evaluate respondents' proposals 


Attachment A 


A Long-Term Assessment Plan for Indiana: Driving 
Student Learning 


Attachment B 


M/WBE Participation Plan Form 


Attachment C 


Sample Contract 


Attachment D 


Indiana Economic Impact Form 


Attachment E 


Formatted Cost Sheet 
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1.6 QUESTION/INQUIRY PROCESS 

All questions or inquiries regarding this RFP must be submitted in writing by the 
deadline of 3 p.m. Eastern Time on April 10, 2007. Questions or Inquiries may be 
submitted via email to rfp@idoa.lN.Eov and must be received by the Procurement 
Division by the time and date indicated above. 

Following the question and inquiry due date. Procurement Division personnel will 
compile a list of the questions and inquiries submitted by all Respondents. The responses 
will be posted to the IDOA website according to the RFP timetable established in Section 
1 .23. The question or inquiry and answer link will become active after responses to all 
questions have been compiled. Only answers posted on the IDOA website will be 
considered official and valid by the State. No Respondent shall rely upon, take any 
action, or make any decision based upon any verbal communication with any State 
employee. 

Inquiries are not to be directed to any staff member of IDOE or to any member of the 
State Board. Such action may disqualify Respondent from further consideration for a 
contract resulting from this RFP. 

If it becomes necessary to revise any part of this RFP, or if additional information is 
necessary for a clearer interpretation of provisions of this RFP prior to the due date for 
proposals, an addendum will be posted on the IDOA website. If such addenda issuance is 
necessary, the Procurement Division may extend the due date and time of proposals to 
accommodate such additional information requirements, if required. 

No Contact: Any contact with the State concerning this RFP should be limited (and, if 
possible, should be in writing) to the DOA's point of contact. Erin Kremer is designated 
as the sole point of contact. The ISBE specifically' instructs that no contact concerning 
this RFP be made with any member of the ISBE, the IDOE, or their respective 
employees, during the selection process. Failure to meet this requirement may disqualify 
Respondent from further consideration for contract resulting from this RFP. 

1.7 DUE DATE FOR PROPOSALS 

All proposals must be received at the address below by the Procurement Division no later 
than 3 pjn. Eastern Time on April 30, 2007. Each Respondent must submit one original 
(marked "Original") and twelve (12) complete copies of the proposal, including the 
Transmittal Letter and other related documentation as required in this RFP. A complete 
copy of the proposal must be provided on CD-ROM. Each copy of the proposal must 
follow the format indicated in Section Two of this document. Unnecessarily elaborate 
brochures or other presentations, beyond those necessary to present a complete and 
effective proposal, are not desired. All proposals must be addressed to: 
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Indiana Department of Administration 
Procurement Division 
Attention: Erin Kremer 
402 West Washington Street, Room W478 
Indianapolis, IN 46204 

Caution to Respondents about shipping/mailing: United States Postal Express and 
Certified Mail are both delivered to the Government Center Central Mailroom, and not 
directly to the Procurement Division. It is the responsibility of the Respondent to make 
sure that solicitation responses are received by the Procurement Division at the 
Department of Administration's reception desk on or before the designated time and date. 
Late submissions will not be accepted. The Department of Administration, Procurement 
Division clock is the official time for all solicitation submissions. 

All proposal packages must be clearly marked with the RFP number, due date, and time 
due. Any proposal received by the Department of Administration, Procurement Division 
after the due date and time will not be considered. Any late proposals will be returned, 
unopened, to the Respondent upon request. All rejected proposals not claimed within 30 
days of the proposal due date will be destroyed. 

No more than one proposal per Respondent per Component may be submitted. 

The State accepts no obligations for costs incurred by Respondents in anticipation of 
being awarded a contract. 

All proposals submitted to the State should be double-sided and printed on 30% post- 
consumer recycled content paper or tree-free paper. When possible, soy ink should be 
used. 

1.8 PRE-PROPOSAL CONFERENCE 

A p re-proposal conference will be held on April 5, 2007 at 1:00 PM in Conference 
Room 1 of the Indiana Government Center Complex, South Building. At this 
conference, potential respondents may ask questions about the RFP and the RFP process. 
Respondents are reminded that no answers issued verbally at the conference are binding 
on the State and any information provided at the conference, unless it is later issued in 
writing, also is not binding on the State. 

1.9 MODIFICATION OR WITHDRAWAL OF OFFERS 

Modifications to responses to this RFP may only be made in the manner and format 
described in Section 1 .6 and clearly identified as a modification. 

A Respondent's authorized representative may withdraw the proposal, in person, prior to 
the due date. Proper documentation and identification will be required before the 
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Procurement Division will release the withdrawn proposal. The Respondent's authorized 
representative will be required to sign a receipt for the withdrawn proposal. 

Modification to, or withdrawal of, a proposal received by the Procurement Division after 
the exact hour and date specified for receipt of proposals will not be considered. 

1.10 PRICING 

Pricing on this RFP must be firm and remain open for a period of not less than 180 days 
from the proposal due date. 

Please refer to the Formatted Cost Sheet, Attachment E, and Section 2.5 for a detailed 
discussion of the proposal pricing format and requirements. 

1.11 PROPOSAL CLARIFICATIONS AND DISCUSSIONS. 
AND CONTRACT DISCUSSIONS 

The State reserves the right to request clarifications on proposals submitted to the State. 
The State also reserves the right to conduct proposal discussions, either oral or written, 
with Respondents. These discussions could include requests for additional information, 
requests for cost or technical proposal revision, clarification as to methods and means to 
provide services, confirmation of satisfaction of required terms, etc. Additionally, in 
conducting discussions, the State may use information derived from proposals submitted 
by competing Respondents only if the identity of the Respondent providing the 
information is not disclosed to others. The State will provide equivalent information to 
all Respondents that have been chosen for discussions. Discussions, along with 
negotiations with responsible Respondents, may be conducted for any appropriate 
purpose. 

Hie Procurement Division will schedule all discussions. Any information gathered 
through oral discussions must be confirmed in writing. 

It is the State's expectation that any material elements of the contract will be substantially 
finalized prior to contract award. Common provisions required by 1DOA and expected to 
be in any resulting contract are provided in Attachment C. (See Section 2.3.5 for details.) 
Any requested changes to these provisions must be submitted with the Respondent's 
response. Hie State reserves the right to reject any of these requested changes. 

1.12 BEST AND FINAL OFFER 

The State may request best and final offers from those Respondents determined by the 
State to be reasonably viable for contract award. However, the State reserves the right to 
award a contract on the basis of initial proposals received. Therefore, each proposal 
should contain the Respondent's best terms from a price and technical standpoint 
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Following evaluation of the best and final offers, the State may select for final contract 
negotiations or execution the offers that are most advantageous to the State, considering 
costs and the evaluation criteria in this RFP. 

1.13 REFERENCE SITE VISITS 

The State may request a visit to any sites to be utilized in execution of the work proposed 
in Respondent's response and proposal to aid the State in the evaluation of the 
Respondent's proposal. Site visits, if required, will be discussed in the technical 
proposal. 

1.14 TYPE AND TERM OF CONTRACT 

The State intends to sign a contract with one or more Respondent(s) to fulfill the 
requirements in this RFP. 

The term of the contract shall be for a period of two (2) years from the date of contract 
execution. There may be two (2), two (2) - year renewals for a total of six (6) years at the 
State's option. 

1.15 CONFIDENTIAL INFORMATION 

Respondents are advised that materials contained in proposals are subject to the Access to 
Public Records Act (APRA), IC 5-14-3 et seq., and, after the contract award, the entire 
RFP file may be viewed and copied by any member of the public, including news 
agencies and competitors. Respondents claiming a statutory exception to the APRA must 
place all confidential documents (including the requisite number of copies) in a sealed 
envelope clearly marked "Confidential" and must indicate in the Transmittal Letter and 
on the outside of that envelope that confidential materials are included. The Respondent 
must also specify which statutory exception of APRA that applies. The State reserves the 
right to make determinations of confidentiality. If the Respondent does not identify the 
statutory exception, the Procurement Division will not consider the submission 
confidential. If the State does not agree that the information designated is confidential 
under one of the disclosure exceptions to APRA, it may seek the opinion of the Public 
Access Counselor. Prices are not confidential information. 

1.16 TAXES 

■ Proposals should not include any tax from which the State is exempt 

1.17 PROCUREMENT DIVISION REGISTRATION 

In order to receive an award, you must be registered as a bidder with the Department of 
Administration, Procurement Division. Therefore, to ensure there is no delay in the 
award, all Respondents are strongly encouraged to register prior to submission of their 
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response. Respondents should go to www.in.gov/idoa/proc and click on "Bidder 
Registration" to register. 

1.18 SECRETARY OF STATE REGISTRATION 

If awarded the contract, the Respondent will be required to register, and be in good 
standing, with the Secretary of State. Hie registration requirement is applicable to all 
limited liability partnerships, limited partnerships, corporations, S -corporations, nonprofit 
corporations and limited liability companies. Information concerning registration with 
the Secretary of State may be obtained by contacting: 

Secretary of State of Indiana 
• . Corporation Division 

402 West Washington Street, EO 18 
Indianapolis, IN 46204 
(317)232-6576. 
www.in.gov/sos 

1.19 COMPLIANCE CERTIFICATION 

Responses to this RFP serve as a representation that the Respondent has no current or 
outstanding criminal, civil, or enforcement actions initiated by the State, and it agrees that 
it will immediately notify the State of any such actions. The Respondent also certifies 
that neither it nor its principals are presently in arrears in payment of its taxes, permit fees 
or other statutory, regulatory or judicially required payments to the State. The 
Respondent agrees that the State may confirm, at any time, that no such liabilities exist, 
and, if such liabilities are discovered, that the State may bar the Respondent from 
contracting with the State, cancel existing contracts, withhold payments to setoff such 
obligations, and withhold further payments or purchases until the entity is current in its 
payments on its liability to the State and has submitted proof of such payment to the 
State. 

1.20 EQUAL OPPORTUNITY COMMITMENT 

Pursuant to IC 4-13-16.5 and in accordance with 25 IAC 5, it has been determined that 
there is a reasonable expectation of minority and woman business enterprises 
subcontracting opportunities on a contract awarded under this RFP. Therefore contract 
goals of five percent (5%) for Minority Business Enterprises and five percent (5%) for 
Woman Business Enterprises have been established and all Respondents will be expected 
to comply with the regulation set forth in 25 IAC 5. 

Failure to meet these goals will affect the evaluation of the proposal. 
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1.21 MINORITY & WOMEN'S BUSINESS ENTERPRISES RFP 
SUBCONTRACTOR COMMITTMENT 

In accordance with 25 IAC 5-5, the respondent is expected to submit with its proposal a 
MWBE Subcontractor Commitment Form. The Form must show that there are, 
participating in the proposed contract. Minority Business Enterprises (MBE) and Women 
Business Enterprises (WBE) listed in the Minority and Women's Business Enterprises 
Division (MWBED) directory of certified firms located at www.buvindinana.in.Eov . 
If participation is met through use of vendors who supply products and/or services 
directly to the Respondent, the Respondent must provide a description of products and/or 
services provided that are directly related to this proposal and the cost of direct supplies 
for this proposal. Respondents must complete the Subcontractor Commitment Form in 
its entirety. Each Respondent most submit separate MWBE Subcontractor 
Commitment forms for each component included in Respondent's proposal. The 
information reported by the Respondent within the MWBE Subcontract 
Commitment form should be exclusive to that component's MWBE Subcontractor 
Commitment Instructions for identifying the baseline for commitment percentage 
calculation is included in Attachment E, Formatted Cost Sheet 

Failure to meet these goals will affect the evaluation of your Proposal. The Department 
reserves the right to verify all information included on the MWBE Subcontractor 
Commitment Form. 

Respondents are encouraged to contact and work with MWBED at 317-232-3061 to 
design a subcontractor commitment to meet established goals as referenced in this 
solicitation. 



Prime Contractors must ensure that the proposed subcontractors meet the 
following criteria: 

• Must be listed on the IDOA Directory of Certified Firms 

• Each firm may only serve as one classification - MBE or WBE 

• A Prime Contractor who is an MBE or WBE must meet subcontractor goals 
by using other listed certified firms. Certified Prime Contractors cannot count 
their own workforce or companies to meet this requirement 

• Must serve a commercially useful function. The firm must serve a value- 
added purpose on the engagement. 

• Must provide goods or service only in the industry area for which it is 
certified as listed in the directory at w ww . bu vindiana. in. gov 

• Must be used to provide the goods or services specific to the contract 

• National Corporate Diversity Plans are generally not acceptable 
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MINORITY & WOMEN'S BUSINESS ENTERPRISES RFP SUBCONTRACTOR 

LETTER OF COMMITMENT 



One or more signed letters), on company letterhead, from the MBE and/or WBE must 
accompany the MWBE Subcontractor Commitment Form. Each letter shall state and will 
serve as acknowledgement from the MBE and/or WBE of its subcontract amount, a 
description of products and/or services to be provided on this project, and reference the 
component and approximate date the subcontractor will perform work on this contract 
Hie State will deny evaluation points if a required letter is not attached, not on company 
letterhead, not signed and/or does not reference and match the subcontract amount and 
the anticipated period that the Subcontractor will perform work for this solicitation. 

By submission of the Proposal, the Respondent acknowledges and agrees to be bound by 
the regulatory processes involving the State's MAVBE Program. Questions involving the 
regulations governing the MWBE Subcontractor Commitment Form should be directed 
to: Minority and Women's Business Enterprises Division at (317) 232-3061 or 
mwbe @ idoa. i n . gov . 

1.22 AMERICANS WITH DISABILITIES ACT 

The Respondent specifically agrees to comply with the provisions of the Americans with 
Disabilities Act of 1990 (42 U.S.C. 12101 ex seq. and 47 U.S.C. 225). 

1.23 SUMMARY OF MILESTONES 

The following timeline is only an illustration of the RFP process. The dates associated 
with each step are not to be considered binding. Due to the unpredictable nature of the 
evaluation period, these dates are commonly subject to change. At the conclusion of the 
evaluation process, all Respondents will be informed of the evaluation team's findings. 



Key RFP Dates: 



t " «. ; ? J . ..." ' 1 'Activity^ : \, .* ;f 


v '-'vbate:'- K : V, 


Issue of RFP 


April 2, 2007 


Pre-Proposal Conference 


April 5, 2007 


Deadline to Submit Written Questions 


April 10, 2007 


Response to Written Questions/RFP Amendments 


April 17, 2007 


Submission of Proposals 


April 30, 2007 
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Proposal Evaluation 


TBD 


Proposal Discussions/Clarifications (if necessary) 


TBD 


Oral Presentations (if necessary) 


TBD 


Best and Final Offers (if necessary) 


TBD 


Contract Award 


TBD 
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SECTION TWO 
PROPOSAL PREPARATION INSTRUCTIONS 



2.1 GENERAL 

To facilitate the timely evaluation of proposals, a standard format for proposal 
submission has been developed and is described in this section. All Respondents are 
required to format their proposals in a manner consistent with the guidelines described 
below: 

• Each item must be addressed in the Respondent's proposal. 

• The Transmittal Letter must be in the form of a letter. The business and technical 
proposals must be organized under the specific section titles as listed below. 

• A Respondent may submit one or more proposal(s), for one, multiple, or all of the 
components of the Assessment System described in this RFP. If a Respondent 
chooses to submit a proposal for more than one of the components, the Respondent 
must submit a separate proposal for each component of the Assessment System that 
the Respondent desires be considered separately. 

• The State reserves the right to consider and award components individually or 
collectively, based on what it deems in its best interest. Respondents should take this 
into account in developing their proposals. 

• Administrative Contract Manager: Depending upon the complexity of the 
assessment program, the Indiana Department of Education and the State Board may 
elect to conduct a separate solicitation to select an Administrative Contract Manager 
who would become the prime contractor for the entire assessment system. Hie 
responsibility of the prime contractor would be to coordinate the implementation, 
administration and review of the assessment system. The prime contractor would be 
the single point of contact for the IDOE and would be held accountable for the 
component deliverables of the assessment system. If IDOE and the State Board 

- determine it is in its best interest to adopt this model, successful Respondents of this 
RFP may become subcontractors of the prime contractor.. Respondents must 
acknowledge their understanding and acceptance of this potential contract outcome 
in the Transmittal Letter. 

12 TRANSMITTAL LETTER 

The Transmittal Letter must address the following topics, except those specifically 
identified as "optional." 

2.2.1 Agreement with Requirements listed in Section 1 

The Respondent must explicitly acknowledge understanding and acceptance of 
the general information presented in Section 1 and agreement with any 
requirements and conditions listed in Section 1 . 

- 2.2.2 Summary of Ability and Desire to Supply the Required Products or Services 
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The Transmittal Letter must briefly summarize the Respondent's ability to supply 
the requested products and/or services that meet the requirements defined in 
Section Three of this RFP. The letter must also contain a statement indicating the 
Respondent's willingness to provide the requested products and/or services 
subject to the terms and conditions set forth in the RFP including, but not limited 
to, the State's mandatory contract clauses. 

2.2.3 Signature of Authorized Representative 

A person authorized to commit the Respondent to its representations and who can 
certify that the information offered in the proposal meets all general conditions, 
including but not limited to the information requested in Section 2.3.4, must sign 
the Transmittal Letter. In the Transmittal Letter, the Respondent must 
indicate the Respondent's principal contact for the proposal along with a 
current and effective mailing and overnight mail address, telephone and fax 
number as well as an e-mail address, if that contact is different than the 
individual authorized for signature. 

2.2.4 Respondent Notification 

Unless otherwise indicated in the Transmittal Letter, Respondents will be notified 
via e-mail. 

It is the Respondent's obligation to notify the Procurement Division of any 
changes in any address that may have occurred since the origination of this 
solicitation. The Procurement Division will not be held responsible for incorrect 
vendor/contractor addresses. 

2.2.5 Other Information 

This item is optional. Any other information the Respondent may wish to briefly 
summarize will be acceptable. 

23 BUSINESS PROPOSAL 

Trie Business Proposal must address the following topics, except those specifically 
identified as "optional." 

2.3.1 General (optional) 

This section of the business proposal may be used to introduce or summarize any 
information the Respondent deems relevant or important to the State's successful 
acquisition of the products and/or services requested in this RFP. 

2.3.2 Respondent's Company Structure 
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The legal form of the Respondent's business organization, the state in which 
formed (accompanied by a certificate of authority), the types of business ventures 
in which the organization is involved, and a chart of the organization are to be 
included in this section. If the organization includes more than one product 
division, the division responsible for the development and marketing of the 1 . 
requested products and/or services in the United States must be described in more 
detail than other components of the organization. If the organization is the parent 
or the subsidiary of or is otherwise related to another organization, those related 
organizations should be identified as well. 

2.3.3 Company Financial Information 

This section must include the Respondent's financial statement, including an 
' income statement and balance sheet, for each of the two most recently completed 
fiscal years. Hie financial statements must demonstrate the Respondent's 
financial stability. If the financial statements being provided by the Respondent 
are those of a parent or holding company, additional financial information should 
be provided for the entity or organization directly responding to this RFP. 

2.3.4 Integrity of Company Structure and Financial Reporting 

This section must include a statement indicating that the Respondent's CEO 
and/or CFO has taken personal responsibility for the thoroughness and correctness 
of any and all financial information supplied with this proposal. The particular 
areas of interest to the State in considering corporate responsibility include the 
following items: separation of audit functions from corporate boards and board 
members, if any; the manner in which the organization assures board integrity; 
and the separation of audit functions and consulting services. The State will 
consider the information offered in this section to determine the responsibility of 
the Respondent under IC 5-22- 16- 1(d). 

The Sarbanes Oxley Act of 2002, H.R. 3763, is NOT directly applicable to this 
procurement; however, its goals and objectives may be used as a guide in the 
determination of corporate responsibility for financial reports. 

2.3.5 Contract Terms and Clauses 

A sample contract that the state expects to execute with the successful 
Respondents) is provided in Attachment B. This contract contains both 
mandatory and non-mandatory clauses. Mandatory clauses are listed below and . 
are non-negotiable. Other clauses are highly desirable. It is the State's 
expectation that the final contract will be substantially similar to the sample 
contract provided in Attachment B. 
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In your Transmittal Letter, please indicate acceptance of these mandatory contract 
terms (see section 2.2.2). In this section, please review the rest of the contract and 
indicate your acceptance of the non-mandatory contract clauses. If a non- 
mandatory clause is not acceptable as worded, suggest specific alternative 

wording to address issues raised by the specific clause. If you require additional t 
contract terms please include them in this section. To reiterate, it is the State's 
strong desire to not deviate from the contract provided in the attachment and as 
such the State reserves the right to reject any and all of these requested changes. 

The mandatory contract terms are as follows: 

• Authority to Bind Contractor 

• Duties of Contractor, Rate of Pay, and Term of Contract 

• Compliance with Laws 

• Conflict of Interest 

• Drug-free Workplace Provision and Certification 

• Funding Cancellation 

• Indemnification 

• Governing Laws 

• Non-discrimination clause 

• Payments 

• Penalties/Interest/Attorney's Fees 

• Non-collusion and Acceptance 

• Information Technology 

Any or all portions of this RFP and any or all portions of the Respondent's 
response may be incorporated as part of the final contract. 

2.3.6 References 

The Respondent should include a list (preferably with 3 references) of any states, 
countries or other clients for whom the Respondent has provided products and/or 
services that are the same or similar to those products and/or services requested in 
this RFP. Information provided should include the name, address and telephone 
number of the client facility and the name, title, phone and fax numbers, and e- 
mail address of a person who may be contacted for further information. 

2.3.7 Registration to Do Business 
Secretary of State 

If awarded the contract, the Respondent will be required to be registered, and be 
in good standing, with the Secretary of State. The registration requirement is 
applicable to all limited liability partnerships, limited partnerships, corporations, 
S -corporations, nonprofit corporations and limited liability companies. The 
Respondent must indicate the status of registration, if applicable, in this section of 
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the proposal. 

Department of Administration. Procurement Division 

Additionally, respondents must be registered with the IDOA. This can be 
accomplished online at www.in.gov/idoa/proc . 

The IDOA Procurement Division maintains two databases of vendor information. 
The Bidder registration database is set up for a vendor to register if it is interested 
in selling a product or service to the State of Indiana. Respondents may register 
online at no cost to become a Bidder with the State of Indiana. To complete the 
online Bidder registration, the vendor should go to 

http://www.in.gov/idoa/register/ . The Bidder registration offers email notification 
of upcoming solicitation opportunities, corresponding to the Bidder's area(s) of 
interest selected during the registration process. Respondents do need to be 
registered to bid on and receive email notifications. Completion of the Bidder 
registration will result in your name being added to the Bidder's Database, for 
email notification. The Bidder registration requires some general business 
information, an indication of the types of goods and services you can offer the 
State of Indiana, and location(s) within the state that you can supply or service. 
There is no fee to be placed in Procurement Division's Bidder Database. To 
receive an award, you must be registered as a Bidder. 

Problems or questions concerning the registration process or the registration form 
can be e-mailed to Amey Redding, Vendor Registration Coordinator, at 
aredding@idoa.in.gov, or you may reach her by phone at (317) 234-0234. 

2.3.8 Authorizing Document 

Respondent's personnel signing the Transmittal Letter of the proposal must be 
legally authorized by the organization to commit the organization contractually. 
This section shall contain proof of such authority. A copy of corporate bylaws or 
a corporate resolution adopted by the board of directors indicating this authority 
will fulfill this requirement 

2.3.9 Subcontractors 

The Respondent is responsible for the performance of any obligations that may 
result from this RFP and shall not be relieved by the non-performance of any 
subcontractor. Any Respondent's proposal must identify all subcontractors and 
describe the contractual relationship between the Respondent and each 
subcontractor. Either a copy of the executed subcontract or a letter of agreement 
over the official signature of the firms involved must accompany each proposal. 

Any subcontract(s) entered into by the Respondent must be in compliance with all 
State statutes and will be subject to the provisions thereof. For each portion of the 
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proposed products or services to be provided by a subcontractor, the technical 
proposal must include the identification of the functions to be provided by the 
subcontractor and the subcontractor's related qualifications and experience. 

The combined qualifications and experience of the Respondent and any or all 
subcontractors will be considered in the State's evaluation. The Respondent must 
furnish information to the State as to the amount of the subcontract, the 
qualifications of the subcontractor for guaranteeing performance, and any other 
data that may be required by the State. All subcontracts held by the Respondent 
must be made available upon request for inspection and examination by 
appropriate State officials, and such relationships must meet with the approval of 
the State. 

The Respondent must list the name, address and the state in which formed of any 
subcontractors) that the Respondent proposes to use in providing the required 
products or services. Each subcontractor's responsibilities under the proposal, 
anticipated dollar amount for subcontract, the subcontractor's form of 
organization, and confirmation from the subcontractor of its commitment to carry 
out these responsibilities are to be included for each subcontractor. This assurance 
in no way relieves the Respondent of any responsibilities in responding to this 
RFP or in completing the commitments documented in the proposal. The 
Respondent must indicate which, if any, subcontractors qualify as a Minority or 
Women Owned Business under IC 4-13-16.5-1. See Section 1.21 and Attachment 
B for Minority and Women Business information. 
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2.4 TECHNICAL PROPOSAL 



The Technical Proposal must be divided into the Components as described below. Every 
point made in each section must be addressed in the order given. The same outline 
numbers must be used in the response. RFP language should not be repeated within the 
response. Where appropriate, supporting documentation may be referenced by a page and 
paragraph number. However, when this is done, the body of the technical proposal must 
contain a meaningful summary of the referenced material. The referenced document must 
be included as an appendix to the technical proposal with referenced sections clearly 
marked. If there are multiple references or multiple documents, these must be listed and 
organized for ease of use by the State. 



Overview 



The Indiana IDOA, acting on behalf of the State Board and the IDOE issues this Request 
for Proposals for the components of its K- 1 2 student assessment system. Interested 
Respondents are directed to Attachment A, the Indiana Student Assessment System Plan 
("the Plan"), for background information and guidance as to the overall purpose and 
objectives of the system and its various components. 

Interested Respondents may submit proposals for one, some or all of the components of 
the Plan. The RFP is being issued on a combined basis to enable IDOE to evaluate each . 
proposal in the context of the overall responses and proposals which would be involved 
in constructing the entire system. 

Respondents are encouraged to submit proposals that meet the criteria identified, whether 
they originate from efforts made specifically for Indiana or from work previously done 
for other states or nations, such that the Respondent's costs for development and other 
activity prior to performance may be minimized or eliminated. 

The components are as follows: 



Grades K-2 


Grades 3-8 


Grades 9-12 






5 Post-Secondary Preparation 




3 Summary 


4 Core 40 End-of-Course 


1 Diagnostic 


2 Diagnostic 




6 Student Learning Portfolio 



Components 3 and 4 will be required for all students in tested grades/courses; 
components 1 , 2, 5 and 6 will be voluntary. 



Exhibit D, page 26 of 92 



I 



Dimensions Common to All Components ; 



As part of their responses. Respondents should demonstrate whether and how their 
assessment(s) reinforce(s) these dimensions of the overall system: 

1. PURPOSE 

• Does the assessment: 

o Drive student learning? If so, how? 

o Support high quality instruction for learning? If so, how? 

o Evaluate individual student learning issues or abilities so instruction can 

adjust accordingly? If so, how? 
o Provide continual and periodic data that defines student progress? If so, how? 

2. ALIGNMENT 

• How does the assessment align to and draw from the auricular aims, in the relevant 
content areas at each grade level and in each course, of Indiana's Academic 
Standards? 

• How does the assessment evaluate a manageable number of demonstrated 
competencies and masteries that link the content of the Standards to the development 
of skills needed for post-secondary success? 

• How does the assessment relate or align vertically across grades and learning levels 
and link to Standards and aims at other levels of learning? 

• How does the assessment correlate with externally provided tests such as NAEP, 
TTMMS, PISA, Advanced Placement or other examinations that Indiana could 
participate in and Indiana students could have access to without conflict with 
Indiana's assessments? 

3. CONSISTENCY 

• How will the assessment feed results on an individual basis tied to each student's 
STN? 

• How will the assessment link to others so they build comprehensively into a learning 
portfolio for each student? 

• If applicable, how will the assessment allow crosswalks for continued longitudinal 
use of the prior results of ISTEP+ deemed relevant to future assessments? 

4. USEFULNESS 

• If applicable, how will the assessment meet the APA-NCME-AERA Standards for 
Educational and Psychological Testing or otherwise produce results that are valid and 
reliable and data of useful quality? 

• How will the assessment generate and maintain rigor in its administration, in the 
calculation of its raw results, in any conversion of its raw results to reported results, 
and in the reporting of its results? 
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• Does the assessment enable visibility to the full spiral of possible learning results^ not 
just locked within a single grade level or average course content, so the results are 
relevant to the accelerated, average, and struggling learner alike? If so, how? 

• How will the results of the assessment be reported in a simple, understandable format 
useful to teachers, parents, and community leaders alike? 

• How will the results of the assessment feed and drive the educational decisions in 
each classroom, school and corporation, across the range of perspectives? 

o How will it meet applicable NCLB and P.L. 221 requirements? 

o How will it facilitate closing achievement gaps for individual students and for 

students viewed demographically? 
o How will it drive toward higher educational achievement for all students 

across the full height and breadth of student performance? 

• How will the results of the assessment be valid at the classroom level, to inform and 
guide instruction, to help teachers know - student-by-student and collectively - how 
to adjust instruction to drive learning? 

5. TECHNOLOGY 

• How does administration and grading of the assessment take advantage of developing 
technology? 

• If needed, could the assessment use an online platform with print technology, phased 
in so that over time the process used is an online process to the maximum extent 
possible, eventually migrating to electronic administration as the primary form? If so, 
how? 

• During such a transition, could the assessment instrument be administered primarily 
in electronic form but in paper-and-pencil form as needed? 

• Because assessments that are administered on the computer may pose inappropriate 
challenges for some students with specific disabilities, how would the assessment 
provide alternate forms and testing accommodations as required by state and federal 
law? 

• How will the assessment maintain confidentiality of results for each student? 

6. IMPLEMENTATION 

• On what schedule could the assessment be phased in consistently with logistical, 
developmental, and legal parameters? 

• How will the assessment further the State's interest in continued rigor in expectation, 
shorter time from assessment to result, and reduced cost? 

• Will the Respondent supply capacity to host the assessment and its resulting 
information, with the State's option to continue to ask the vendor to host or to shift 
hosting to the State over time? If so, how? 

• If applicable, how will the assessment provide the bank of items by grade and subject 
area, and will the assessment enable the State to add questions to an Indiana-specific 
bank of items by grade and subject area? 

7. IDOE SUPPORT AND ASSISTANCE 
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• How will the assessment enable the IDOE to assist and support local corporations 
through collection and data analysis of the results of the assessments and related 
support for schools? 

• Is the Respondent willing to have the State or another Respondent play a "managing 
partner" role, in partnership with other Respondents? 

7. SPECIAL CONSIDERATIONS 

• For students with disabilities and English language learners, will the assessment 
support classroom educators to apply appropriate accommodations? 



Technical Requirements Potentially Applicable To Components 



Each of the following technical requirements has potential application to one or more of 
the Components. Respondents must acknowledge their understanding and acceptance of 
the applicable listed technical requirements and should identify whether and, if so, how 
their proposal would satisfy the requirements. Respondents must also provide a narrative 
for each applicable requirement that demonstrates its ability to meet the stated 
requirement Requirements 1. through 5. address design elements and their applicability 
will vary across components (for example, diagnostic versus summary, NCLB versus 
post-secondary preparation); 6. through 1 1. address access and reporting elements and 
their applicability is assumed. Hie State will consider proposals in light of explanations 
how one or more technical requirements are or should be inapplicable in the context of a 
particular component or may be satisfied in a manner that meets the State's objectives but 
does not address the requirement literally. Requirements 10. and 1 1 . are non-negotiable. 

1 . All products, assessments and assessment components must meet or exceed 
applicable APA-NCME-AERA Standards for Educational and Psychological 
Testing. In all areas not addressed by 'The Standards," products must meet 
existing "industry standards." 

2. All assessments will be construed following the principals of "Universal Design" 
and will be available in all IDOE-approved accommodated formats. 

3. Unless specifically agreed to by the IDOE, all components of the assessment 
system will be built in a native online environment and "ported" to paper if 
needed so that when sufficient capacity is available within Indiana schools, 
transition to online assessment will be facilitated. 

4. Unless specifically agreed to by the IDOE, all item passages, text, ancillaries, 
images and graphics developed for any of the components are the property of the 
State of Indiana and the IDOE. 

5. All items developed, selected and used for assessments in the content areas of 
mathematics, science, and social studies must be reviewed to ensure they are the 
most direct possible measure of the trait and free from extraneous wording and 
context Any "reading" demand in these content areas must be appropriately 
below (not on or above) grade level. 

6. For all levels of users, any online/electronic access to assessments, components, 
reports, scheduling or ordering shall be via a single secure common interface that 
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will allow access to any information for which a user has permission A parent, a 
teacher or a superintendent will need only a single User ID and Password to 
access any and all appropriate parts of the system. 

7. Whenever possible and practical, reporting will be "electronic." 

8. All data from all components of the assessment system will be delivered in a way 
that will seamlessly integrate with Indiana's data system. 

9. AH respondents will use Indiana's system for identifying students, teachers, 
schools and school corporations. 

10. All proposals will contain a report of all errors that have occurred in the past two 
years for any testing program that assesses more than 5,000 students per grade 
level under a single contract. Errors that are in whole or in part the responsibility 
of the Respondent or its subcontractors must be reported. Error is defined here in 
its broadest sense, meaning any event that results in any incorrect/incomplete 
results being reported or any delay in meeting the scheduled delivery of any 
component of the testing program. Hie report will include the following 
information: 

1. Description of the error, 

2. Scope of the error (number of schools, students, etc.); 

3. Identification of who discovered the error (vendor, customer, site, etc.); 

4. Short-term resolution; 

5. - System changes put into place to prevent future occurrences; and 

6. Customer contact, address and phone number of an individual who can 
address the impact of the error on the testing program. 

Disclosures of errors should not contain any proprietary, FERPA-protected, or 
secure information, but must fully and completely report on the issues described. 

1 1. The successful Respondent will promptly report any errors occurring with other 
programs throughout the duration of this contract. A summary report must be 
submitted at least once every three months beginning with the date this contract is 
enacted. Failure to disclose this information is grounds for termination of any 
contract resulting from this RFP. 
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2.4.1.1 Background Information 

As part of the Plan, the State would make Kindergarten through grade 2 diagnostic 
assessments available for teachers, schools and corporations to use to identify individual 
student learning progress or issues. 

2.4.1.2 History 

Indiana does not currently provide any diagnostic assessments or tools for teachers or 
schools to use in grades K-2 beyond the Reading Assessments currently available. Some 
individual corporations have adopted or developed other K-2 literacy and numeracy 
assessments at their own expense. -' _ 

2.4.13 Elements 

The diagnostic assessments for grades K-2 should focus on the child's ability to read, 
comprehend, and use numbers. 

The assessments are solely diagnostic and will not be part of an accountability system. 
They should inform teachers' understanding of each student's individual circumstances in 
such a manner as to aid the teachers in improving the student's learning. 

The assessments should: 

• be user-friendly for classroom teachers; 

• be classroom based; 

• focus on identifying individual learning needs and issues; 

• provide age-appropriate inquiries that identify a student's level of literacy, numeracy 
and comprehension; 

• enable effective conclusions about students that can be translated into child -specific 
activities; 

• enable real-time reporting to teachers and collection of data for teachers and schools 
to use longitudinally in measuring and identifying student progress, individually and 
collectively; 

• provide a structure and process for teachers to share measurement and evaluation 
techniques that can be embedded in instruction; and 

• inform professional development and be supported by accessible professional 
development resources to improve student literacy and numeracy skills. 

Hie assessments should be effective for all children, such that their usage and results 
inform and assist further progress for the advanced and gifted student, the student who is 
ready, and the student whose learning reflects gaps in preparation, readiness and learning. 
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Hie assessments should be technology enabled. They should provide means for the 
electronic collection of results for use by teachers, schools and corporations and enable 
them to submit data from the assessments to the State, to be used for individual student 
learning record purposes, including student portfolios, and to support research, but not for 
accountability purposes. 

The results of such assessments should be able to be included as a part of the student's 
learning portfolio. 

Hie assessments should provide adequate data such that during the student's K-2 years 
and at the end of second grade, each student's basic literacy, numeracy, and 
comprehension can be determined. 

2.4.1.4 Technical Proposal 

■The IDOE seeks proposals from Respondents regarding the development, administration, 
scoring, and collateral activities of diagnostic assessments for literacy, numeracy and 
comprehension in Kindergarten through grade 2. 

This RFP seeks proposals for technology-enabled administration of such diagnostic 
assessments, consistent with the elements described above. 

This RFP seeks proposals for technology-enabled administration of such diagnostic 
assessments, consistent with the elements described above. 

The work plan should permit the Respondent to conduct the necessary preliminary 
activities involved in preparation for the availability and use of the diagnostic 
assessments. The IDOE seeks a workable, teacher-friendly, long-term approach to the 
assessments consistent with the overall assessment system parameters in the Plan. The 
IDOE has provided some specificity about its needs, but Respondents are welcome to 
make suggestions and suggest alternative strategies or approaches (including those 
already in use elsewhere) that could lead to a more effective and efficient assessment 
process. 

The Respondent must identify: 

• how the diagnostic assessments identify student literacy, numeracy and 
comprehension in grades K-2; 

• how the assessment results confirm preparation for further learning; 

• how the assessments diagnose student learning issues; 

• whether the diagnostic assessments correlate to Indiana's Standards for Kindergarten 
through grade 2 and, if so, how; 

• whether schools will be able to utilize the results of the assessments to benchmark 
student results against other indicators of student performance; and 

• whether the assessments demonstrate validity and reliability for their intended use, in 
the context of their being solely diagnostic and not summative or high-stakes 
assessments and, if so, how. 
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The Respondent should note whether the assessment results can be collected and used 
longitudinally to provide individual teachers feedback that is performance-based and may 
be used to adjust instruction to improve such student achievement; and whether they may 
be included in individual student learning portfolios and in longitudinal studies of student 
performance within schools and across the state. 

The IDOE requires the Respondent to provide all services necessary to support the use of 
these assessments in the classroom environment. 

Hie diagnostic assessments must be available to all students, and the Respondent must 
propose how the assessment mechanism provides appropriate accommodations to all 
students. 

The Respondent should identify how the assessment content will be treated; whether the 
assessment content will remain the property of the Respondent or become the property of 
the State of Indiana; and how the Respondent will grant appropriate license and access to 
the assessment mechanisms and materials. Where the Respondent proposes to utilize 
assessment forms, item banks or other materials developed elsewhere, the Respondent 
must grant appropriate and perpetual license and access to such materials. 

Hie Respondent must be responsible for all operational and support tasks associated with 
administering the diagnostic assessments (whether the assessments are administered in a 
technology-enabled environment or in paper-and-pencil format or both),, and the 
Respondent should explain how its assessments are intended to be deployed, 
administered, scored, utilized, preserved and otherwise used in a classroom setting to 
inform instruction and drive student progress in literacy, numeracy and comprehension. 

The Respondent should demonstrate how the assessments and their results (whether the 
assessments are administered in a technology-enabled environment or in paper-and- 
pencil format or both) will be recorded and retained at the classroom and school level, 
and how they will deliver aggregate score reports to the IDOE. The IDOE prefers a 
technology-enabled system that generates printer-friendly and user-relevant reports at the 
student, classroom, school, school corporation, and state level. 

All functions of any online system, if one is used by the Respondent, must be platform, 
operating system, and browser compatible and provide data in formats compatible with 
recent versions of Netscape Navigator, Internet Explorer, Firefox, Mozilla, Safari, and 
Linux, or the Respondent must present an alternate plan to address the various platforms, 
operating systems, and browsers used by schools. 

The Respondent should demonstrate how quickly the individual performance will be 
reported. The IDOE prefers technology-enabled assessment results be immediately 
available to the classroom teacher and captured at that same time for aggregate use and 
longitudinal reporting. Students using paper-and-pencil assessments must receive score 
reports within 7 calendar days, starting from receipt of the assessment by the Respondent. 
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2.4.1.5 Cost Proposal 



For purposes of planning and budget preparation, the IDOE expects the number of tests 
administered to vary across schools. For purposes of the Respondent's cost proposal, the 
Respondent should plan for and assume the quantities designated on the formatted cost 
sheet (fee schedule), Attachment E. The IDOE prefers a cost proposal that includes a 
cost structure based on the number of assessments actually administered. 
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2.4.2 Component 2: Grade 3 through Grade 8 Diagnostic Assessments 



2.4.2.1 Background Information 

As part of the Plan, the State would make grade 3 through grade 8 diagnostic assessments 
available for teachers, schools and corporations to use to identify individual student 
learning progress or issues in English/language arts, mathematics, science and social 
studies. Such tests would provide classroom teachers with real-time means to determine 
student progress in each area, such that the teacher's instruction and the student's in- year 
and in-subject learning plan could be modified to build on strengths and eliminate 
weaknesses. 

2.4.2.2 History 

Indiana does not currently provide any diagnostic assessments or tools for teachers or 
schools to use in grades 3 through 8. Some individual corporations have adopted or 
developed such assessments at their own expense. 

2.4.23 Elements 

The in- year diagnostic assessments should provide a tool to assist classroom teachers in 
understanding individual student progress within each applicable subject area as the year 
progresses and thus inform their instruction to drive accelerated student learning across 
advanced, on-pace and struggling students. They should provide immediate and specific 
feedback, preferably with suggestions for instruction, as part of every individual student 
report, and thus be able to be embedded in classroom practice, as an approach to teaching 
rather than just a test instrument 

These in-year diagnostic tools should identify individual learning progress, enabling 
effective conclusions about students that can be translated into activity that further 
encourages the advanced and gifted child, the student who is ready, and the student 
whose learning reflects gaps in preparation and readiness. 

The diagnostic assessments must be aligned with state Standards and curricular aims, 
reflecting the substance and rigor of defined subject matter consistent with applicable 
Indiana Standards, such that they induce confidence that results on the diagnostic 
assessments will predict results on the summative assessments. 

The classroom teacher should be able to apply the assessments and results to assist 
students working well above or below, as well as those tracking to grade level 
expectations, such that the assessments should measure across the full scale of learning, 
both above and below grade level (i.e., be grade-less). They should provide diagnostic 
information (i.e., so teachers know why the student got the question wrong) and provide 
benchmarks to identify students who are ahead of, on track for, or lagging behind 
meeting end-of-year expectations for each grade. The reported results of the assessment 
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should enable easy use by the teacher in conferring with the student's parents about the 
student's progress. 

The assessments should be technology enabled. They should be able to be administered 
electronically and online. 

Because the assessments will be locally managed, and because their use will be 
determined as desired at the local level, with the administration frequency - spot use, 
organized periodic use, and/or mid-year interim evaluation, and varying among subject 
areas - as local decisions, the assessments should be flexible in use, available as needed 
by the teacher for whole classroom or individual student application, across the school 
year. 

Individual and collective results should be reportable and used as desired by the local 
classroom, school and corporation. The assessments should enable the collection of 
student performance data for longitudinal visibility to each student's growth in each area 
during the school year, continuity of that visibility as a student passes from teacher to 
teacher, and collective visibility of the student's performance during a year by all the 
teachers responsible for the student's learning. 

The results should be reportable to the student's learning portfolio and thus enable 
(although not require) cross-classroom and teacher community of practice use of 
diagnostic results during each student's year, along with other indicators of student 
performance (portfolios, specific assistance, etc.). The assessment reporting mechanism 
also should enable local school corporations to submit data from these assessments to the 
IDOE, to be used for analysis, support and research purposes but not for accountability 
purposes. 

The assessment may, if feasible, identify student writing skills, but schools also will be 
free to use diagnostic means of their own choosing to assess student writing during the 
school year. 

2.4.2.4 Technical Requirements 

Respondents must acknowledge their understanding and acceptance of the listed technical 
requirements. Respondents must also provide a narrative for each requirement that 
demonstrates its ability to meet the stated requirement 

NOTE: The following identify IDOE expectations for a system of diagnostic 
assessments; however, Respondents are encouraged to offer any solution that will address 
Indiana's desire for a set of tools for ongoing classroom assessment of student 
achievement 

Indiana will consider the use of existing assessments, provided the Respondent 
demonstrates that such will meet all of the goals of Indiana's system and represent a 
significant ultimate time and/or cost savings. Additionally, Respondents may offer 
proposals in which either the Respondent or the State would host the diagnostic 
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assessments. 

The diagnostic assessments proposed by the Respondent should address the following 
requirements: 

• Provide evidence of alignment with Indiana Standards/Indicators; 

• Ensure the State of Indiana has ownership of the item bank; 

• Deliver a Technical Report to the IDOE; 

• Deliver an Annual Report to the IDOE; 

• Report all results to the EDOE; and 

- • Provide professional development to teachers and other educators on 
the use of results to improve learning. 

2.4.2.4.A Alignment with Indiana Standards and Indicators 

Items to be used on the diagnostic assessments must align to and measure performance 
against the Indiana Standards. The items must also be fair and free of bias to ensure that 
the diagnostic assessments provide equitable measures for students with alternative 
cultural and ethnic backgrounds and diverse learning styles. The Respondent should 
propose a process whereby all items for potential use on the diagnostic assessments are 
available to be reviewed and approved by the DX>E. TTie details of this process will be 
finalized in collaboration between the successful Respondent and the IDOE. The 
Respondent will bear the burden of demonstrating alignment with Indiana standards. If it 
chooses, the Respondent may have access to the Indiana subject matter committees who 
use the Indiana Standards as the foundation of their work. 

2.4.2.4J* Item Ownership 

The Respondent is responsible to provide DOE access to and use of the items (through 
perpetual license or ownership). 

2.4.2.4.C Quality Control 

The Respondent is responsible for maintaining high quality control over all testing items 
and rubrics, data entry, processing, and training. 

The Respondent should propose a plan for how it expects to complete all work associated 
with this task, including descriptions of procedures, supporting rationale for procedures, 
and costs. The Respondent should provide evidence of capability and experience in 
providing the services specified under this heading, and of having completed work 
similar to that specified in this RFP, using procedures similar to those required for these 
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tasks. 



The Respondent should submit to the IDOE, as part of the project procedures, the 
documentation of quality control procedures followed by the Respondent to ensure the 
integrity and accuracy of items and rubrics, scoring, reporting, and other aspects of the 
diagnostic assessments. 

2.4.2.4.D Professional Development 

Training on a diagnostic/formative assessment system should go beyond training 
educators to use the system and include how to use the results to immediately impact 
instruction. The Respondent should identify whether and, if so, propose how, it will 
provide access to assistance for training corporation and school personnel in how to use 
the system, the scoring rubrics, the scoring process, how to interpret the results and how 
to make any needed adjustments to instruction. 

2.4.2.4 X Technical Report 

The Respondent will prepare a technical report, in hard copy and electronic formats, to 
provide documentation of all technical and statistical work associated with the 
development of the web-based diagnostic assessment system. The report will include 
detailed narrative descriptions for all components of the online assessment system. It will 
also provide sufficient information to allow for an independent evaluation of the quality 
of the web-based diagnostic assessment system. The final report will be subject to 
approval by the IDOE. 

2.4.2.4 .F Annual Report 

At the end of each school year, a report detailing all aspects of usage of the diagnostic 
assessment system will be prepared and submitted to the IDOE. The report will contain, 
- but not be limited to: analyses by item, student, classroom, school, corporation, content, 
Standard and Indicator. 

' 2.4JL5 Technical Proposal 

The IDOE seeks proposals from Respondents regarding the development, administration, 
scoring, and collateral activities of diagnostic assessments for English/language arts, 
mathematics, science and social studies in grades 3 through 8. 

This RFP seeks proposals for technology-enabled administration of such diagnostic 
assessments, consistent with the elements described above. 

The IDOE requires Respondents to propose a detailed work plan that accomplishes the 
stated objectives. Refining this work plan with the IDOE will be an obligation of the 
successful Respondent, and the revised work plan will be incorporated into the final 
contract documents with the Respondent 
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The work plan should permit the Respondent to conduct the necessary preliminary 
activities involved in preparation for the availability and use of the diagnostic 
assessments. The IDOE seeks a workable, long-term approach to the assessments 
consistent with the overall assessment system parameters in the Plan. The IDOE has 
provided some specificity about its needs, but Respondents are welcome to make 
suggestions and suggest alternative strategies or approaches (including those already in 
use elsewhere) that could lead to a more effective and efficient assessment process. 

The Respondent must identify: 

• how the diagnostic assessments identify student progress in writing and in knowledge 
and skills in English/language arts, mathematics, science and social studies in grades 
3-8; 

• whether the assessments diagnose student learning issues and, if so, how; 

• how the assessments correlate to Indiana's curricular aims and Standards in 
English/language arts, mathematics, science and social studies in grades 3 through 8; 

• whether schools will be able to utilize the results of the assessments to benchmark 
student results against other indicators of student performance; and 

• how the assessments demonstrate validity and reliability for their intended use, in the 
context of their being diagnostic and not summative or high-stakes assessments. 

Hie Respondent should demonstrate how the assessment results can be collected and 
used longitudinally to provide feedback that is performance-based and may be used in 
adjusting instruction to improve such student achievement, and how they may be 
included in individual student learning portfolios and in longitudinal studies of student 
performance within schools and across the state. 

Hie IDOE requires the Respondent to provide all services necessary to support use of 
these assessments in the classroom environment 

The diagnostic assessments must be available to all students, and the Respondent must 
propose how the assessment mechanism provides appropriate accommodations to all 
students. 

The Respondent should identify how the assessment content will be treated; how Indiana 
will have access to the assessment items (either through perpetual license or ownership); 
and how the Respondent will grant appropriate license and access to the assessment 
mechanisms and materials. Where the Respondent proposes to utilize assessment forms, 
item banks or other materials developed elsewhere, the Respondent must grant 
appropriate and perpetual license and access to such materials. 

The Respondent must be responsible for all operational and support tasks associated with 
administering the diagnostic assessments in a technologically-enabled environment, and 
the Respondent should explain how its assessments are intended to be deployed, 
administered, scored, utilized, preserved and otherwise used in a classroom setting to 
inform instruction and drive student progress. 
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The Respondent should demonstrate how the assessments and their results will be 
recorded and retained at the classroom and school level, and how they will deliver 
aggregate score reports to the DDOE. The IDOE prefers an online system that generates 
printer-friendly and user-relevant reports at the student, classroom, school, corporation, 
and state level. 

All functions of any online system, if one is used by the Respondent, must be platform, 
operating system, and browser compatible and provide data in formats compatible with 
recent versions of Netscape Navigator, Internet Explorer, Fire fox, Mozilla, Safari, and 
Linux, or the Respondent must present an alternate plan to address the various platforms, 
operating systems, and browsers used by schools. 

The Respondent should demonstrate how quickly the individual performance will be 
reported. Hie IDOE prefers a result immediately available to the classroom teacher and 
captured at that same time for aggregate use and longitudinal reporting. 

2.4.2.6 Cost Proposal 

For purposes of planning and budget preparation, the IDOE expects the number of tests 
administered to vary across schools. For purposes of the Respondent's cost proposal, the 
Respondent should plan for and assume the quantities designated on the formatted cost 
sheet (fee schedule), Attachment E. The IDOE prefers a cost proposal that includes a 
cost structure based on the number of assessments actually administered. 
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2.43 



2.43.1 Background Information 

Senate Enrolled Act 1240 asked the State Board and the IDOE to develop a system in 
grades 3 through 10 that: 

■ Reflects a student's proficiency in and mastery of the state's academic standards; 

■ Is, to the greatest extent possible, more concise, less time consuming, and less 
expensive to administer than the current tests while maintaining the current level of 
rigor of the tests; 

■ Provides prompt results to students, parents, and teachers; 

■ Explores all options for timing and use of summative tests, including' giving a 
summative test in the fall or the spring; 

■ Measures individual student growth from school year to school year, 

■ Explores all options for diagnostic tests for use by teachers to support ongoing 
remediation; 

■ Involves a transition to the use of online testing; 

■ Assesses student proficiency in written communication in an effective manner, and 

■ Moves to the use of online assessments for Core 40 subjects. 

Diagnostic assessments in grades 3 through 8 are addressed in Component 2; online 
assessments for Core 40 subjects are addressed in Component 4, as are the NCLB, GQE 
and other aspects of summative assessment in grades 9 and 10. This Component 
addresses the remaining objectives in grades 3 through 8. 

2.43.2 History 

The Indiana Student Test of Educational Progress-Plus (ISTEP+) has been the State's 
summative assessment for grades 3 through grade 9. In the context of otherwise 
applicable provisions of Indiana and federal law, this Component requests proposals 
responsive to the objectives identified above, consistent with the elements, requested 
services and technical requirements identified below. 

The IDOE has been careful to maintain compliance with applicable NCLB requirements 
and to develop a valid and reliable assessment approach for grades 3 through 8. As noted 
in the Plan and further below, the IDOE desires to ensure that any system responsive to 
this RFP provides for adequate and effective transition from the prior system; enables 
cross-walk or other effective utilization of assessment results produced by the prior 
system for prior years; and maintains compliance with applicable state and federal 
requirements. 
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2.433 Elements 



The end-of-year summative assessments for English/language arts, mathematics, science 
and social studies in grades 3 through 8 should contain both a writing assessment and a 
progress assessment. 

All students in grades 3 through 8 should take the writing and progress assessments, with 
appropriate accommodations and alternate assessment instruments to support full access 
for eligible students with disabilities and English language learners. 

The "writing" assessment should assess the student's progress in effective written 
expression. It should use applicable Indiana Standards to create essay and other open- 
ended questions that measure student use of grammar and composition for effective 
writing, higher-order thinking, and applied skills in English/language arts, mathematics, 
science and social studies. 

It should be administered annually as late as possible in the second half of each school 
year, yet at a time such that the reporting of its results is synchronized with the reporting 
of the progress assessment by the end of the school year in an integrated report. 

Each student's writing assessment score should be reported both as a separate score 
focused on the student's writing ability and as a factor creating the total English/language 
arts score. 

The progress assessment should reflect each student's growth in knowledge during the 
school year. It should be administered near the end of the school year, with only a brief 
tum-around time so that scores are received before the end of the school year, annually in 
English/language arts, mathematics, and science and every other year in social studies, 
unless a more frequent assessment is required by state or federal law. 

The progress assessment should measure the curricular aims reflected in the Indiana 
Standards for English/language arts, mathematics, science and social studies. It should 
build cumulatively, within and across subject matter areas, reflecting the developmental 
progressions across grades 3 through 8, with assessment instruments aligned to the 
curricular aims and Standards. It should apply the rigor expected in those curricular aims 
and Standards in the knowledge and skills assessed. 

The administration of progress assessments for each subject area need not be contiguous, 
but they must be concise in time to take, in the context of adequate technical rigor and 
quality necessary to meet federal or other applicable standards. The progress assessment 
should have no open-ended or essay questions but should use other means to assess 
higher-order thinking skills and curricular aims. 

The format of the progress assessments must enable electronic administration and scoring 
and provide compatibility to move to online assessments when such capacity exists. 
Respondents must identify their method of scoring and how the method will produce 
timely, valid, reliable, and cost-effective results. 
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Each student's progress assessment score should be reported as a separate result focused 
on the student's proficiency of the curricular aims reflected in the Indiana Standards. 

For both aspects of the end-of-year summative assessments . Respondents should 
identify their method of scoring the assessment and how it will produce relevant, timely, 
valid, reliable, and cost-effective results. The assessments must be scored uniformly 
across the state, with scoring that expects rigor. 

The end-of-year summative assessments (both writing and progress) must have the 
validity, reliability and other attributes adequate for use in the state's fP.L. 221) and the 
federal government's (NCLB) systems of tracking student performance. Data from 
previous summative assessments should be retained and brought into the system by cross- 
walking results from earlier assessments into the system. Respondents should identify 
how they will transition to their proposed assessments responsive to this Component, 
including how they will achieve retention and use of the longitudinal results from earlier 
assessments. 

The results of the end-of-year summative assessments (both writing and progress) will be 
reported annually, by the end of the school year, in a format that facilitates timely 
communication to parents, teachers and schools. The results will become part of each 
student's learning portfolio, evidencing the student's progress to proficiency in the 
curricular aims. Aggregate results will be reported at the class, school, corporation, and 
state levels. 

The results will be tracked by individual student's STN, so that the results are both 
portable over geography (as the student moves from school to school or corporation to 
corporation) and available over time (as the student rises through grades K-12 to 
graduation and ultimately into post-secondary education). 

2.43.4 Technical Requirements 

Respondents must acknowledge their understanding and acceptance of the listed technical 
requirements. Respondents must also provide a narrative for each requirement that 
demonstrates its ability to meet the stated requirement. 

Background 

The ISTEP statewide assessment system has existed in Indiana since 1989. However, in 
its current mixed-format form, ISTEP+ has been administered since the fall of 1997. 
New Indiana Academic Standards were adopted in 2000 and the tests for grades 3, 6, and 
8 underwent revision, alignment and cut score setting in 2002. In 2004, the Graduation 
Qualifying Exam (GQE) was similarly revised and tests were added at grades 4, 5, 7, and 
9. Science was added to grade S in 2003 and to grade 7 in 2005. 

Currently, ISTEP+ assessments in grades 3 to 10 and content areas (E/la, math 
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and science) include both selected response and constructed response items. Indiana 
owns all items and content (or has permission to use content) on the current assessments 
and has a piloted item pool that will allow for the construction of two future forms of all 
assessments using the existing test blueprints. 

A number of state laws and rules govern the content, construction and delivery of 
ISTEP+. Please fully review Appendix E of the ISTEP+ Program Manual and all of IC 
20-32 before responding to this section of the RFP as every proposal must meet these 
legal requirements. Failure to meet all current laws and rules may disqualify the 
Respondent's proposal from consideration for this Component of the system. (Keep in 
mind, laws are subject to change by the legislature and rules are subject to change by the 
State Board.) 

ISTEP+ is the key student achievement measure for both state and federal accountability. 
Indiana's system of Standards and Assessments has been fully approved by USDOE 
through the "peer review** process required by NCLB. It is critical that all Respondents 
understand that any proposal made for this Component of the assessment system is 
required to, upon federal review, receive the same or higher status or be subject to 
liquidated damages. 

2.43.4 A. Assessment Transitions 

This RFP requires two significant changes to the current ISTEP+ assessment: I ) a 
changing the time of year that the assessment is administered; and 2) splitting the test into 
a "writing" (essay and constructed response) portion and a "progress" (selected response 
and other machine scorable items) portion so that an "end of year" achievement result 
will be determined for every tested student 

These transitions introduce a variety of significant risks into an already complex 
enterprise. Each Respondent must address these issues in a separate subsection of the 
proposal. Even if activites, processes and control systems are integrated within the larger 
system, the IDOE wishes to see specifically how the risks and costs associated with these 
technical transitions will be handled. 

2.43.4A.1 Time of Year 

Specifically, each Respondent must, at a minimum, address the following issues for the 
one-time event — transition in the time of testing: 

(Each Respondent must also include any other procedures that, based on the 
Respondent's experience, are required.) 

• How all current test items will be "recalibrated" to the **new" time of year, 
o Timeline for such recalibration 
o Methodology to be used 

o Sample size needed and proposed distribution and composition of the 
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sample 

o Criteria for determining if any change in item statistics is acceptable 
o How **new" item statistics will be correctly associated with items 

2.43.4.A.2 Format of Test 

Indiana anticipates that the requirements for constructed response items and for 
scoring completed by the end of the school year will necessitate the splitting of the 
test into a "writing" and a "progress" assessment. Each Respondent must identify its 
proposed format and length for the writing and progress portions of the summative 
assessment and should address both the items and issues brought forward by the State 
and any and all other issues that, based oh the Respondent's experience, must be dealt 
with in order to make changes to the test The IDOE is not locked into any specific 
solution and encourages innovative proposals that will meet the State's 
objectives. 

• Determining content/blueprint of the test and impact of test design on content that 
may be assessed 

o Proposed design for "end of year" 
o Proposed blueprint for "writing" test 
o Proposed blueprint for "progress" test 

• A "typical" school year timeline that will show all key assessment events within a 
full cycle of testing, including reporting and NCLB accountability 

• Turn-around time including: 

o QA/cleanup process for demographic data associated with assessment 
scores 

o Delivery of student, class, school, corporation and state results 
o Timeline for required re score process 

o Delivery of final electronic data to state to facilitate accountability 
determinations 

• How missing data will be handled if the test is administered in two separate 
windows 

■ Psychometric "plan" including any anticipated "special" technical reports and 
issues 

o Anomalous student results: will they be identified or flagged? 

o How will cut scores on the new test be determined? 

o Indiana requires a vertical scale that will directly measure student growth 

for the purposes of state accountability; how will it be constructed? 
o How will previous student scores and "status" be equated to scores and 
"status" on the "new" assessments? 

■ How will reliability and validity of these equated scores be 
demonstrated? 
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■ For purposes of Indiana accountability, we must have three years 
of comparable growth data for students. 

• Complete timeline and project plan for moving to end-of-year summati ve 
assessment 

• Plan, costs and timeline to deal with the requirements of NCLB and AYP during 
the transition 

o ISTEP+ is a fall test that assesses the prior year's Standards and 

Indicators. AYP determinations based on the fall 2006 test are for the 
2005-06 school year. 

o During the school year in which the change is made from fall to spring, 
every student in an NCLB-required grade will need to be tested twice. 

o Plan to align new assessment for NCLB purposes 
_ o • Timeline for submission to USDOE for "Peer Review" 



• Total "one time" costs of "moving" the tests 

2.4J.4.B Program Manager and Project Management Team 

The Respondent shall assign a Program Manager, dedicated full-time to this project, and 
assemble a project management team to oversee and coordinate the efforts of the . 
contractor and all related subcontractors. The Program Manager shall serve as the 
primary liaison with the IDOE for all components of the project. The Program Manager 
must have demonstrated previous experience with managing a large, customized 
assessment project similar in scope and nature to the summative assessments. 

A cohesive, dedicated, skilled core team is critical to the summative assessment project. 
At a minimum, this core team of key personnel shall be listed in the Respondent's 
proposal. Personnel commitments and identifying core team key personnel as specific 
individuals are a mandatory requirement for the work to be performed under this contract 
and subject to approval by the IDOE Division of School Assessment. The Respondent 
shall provide an organizational chart showing all key staff and offices assigned to work 
on the various aspects of the summative assessments. Roles and responsibilities for all 
key staff shall be identified. 

Except in the case of illness, death, or leave of absence and so long as the personnel 
remain partners, principals, or employees of Respondent, no re-deployment of any 
member of the core team as required by the work plan may be made without prior written 
consent of the IDOE Division of School Assessment, which shall not be withheld without 
good cause. Replacement of such personnel, if approved, shall be with personnel of 
equal or greater ability and qualifications. 
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2.43.4.C Project Plans and Schedules 

The Respondent shall continuously monitor the ongoing operations of the summative 
assessments using a detailed project plan and schedule. The Respondent shall provide an 
annual updated project plan and schedule for joint use by the IDOE and the Respondent 
in monitoring all program activities. The IDOE desires appropriate direct real time 
access to the project schedule. 

2.43.4.D Status and Planning Meetings 

The Respondent shall participate in weekly status meetings with the IDOE. These 
meetings will be conducted by telephone and/or by pre-arranged videoconferencing, 
Webrex, or via other communication technology as proposed by the Respondent The 
Respondent shall be responsible for providing written documentation of the weekly status 
meetings within 24 hours of each meeting. The Respondent shall also provide weekly 
written status reports that shall include but not be limited to, reports on the status of 
ongoing activities, decisions made, decisions pending, activities completed, problems, 
and timelines for scheduled activities. Weekly status reports shall be in a format that is 
approved by the IDOE Division of School Assessment. 

The Respondent shall participate in and support all associated costs for a minimum of six 
planning meetings of the IDOE and Respondent project management teams, other 
contractor management staff responsible for services provided to the IDOE, and 
representatives of the Respondent's senior management team, as necessary. Three two- 
day meetings will be held at the Respondent's site and three two-day meetings will be 
held in the State of Indiana at a site convenient to the IDOE staff (a total of 12 meeting 
days). The Respondent shall be responsible for all costs associated with these meetings. 
The IDOE staff attending the meetings will include the assistant superintendent, the 
assessment director, and up to four education associates. During the initial "start-up" 
phase of this project there may be a need for additional meetings; these will be at the 
Respondent's expense with dates, times and locations to be mutually agreed upon by the 
Project Manager and the Director of School Assessment. The Respondent shall submit 
the meeting notes to the IDOE within five (5) working days of each meeting for final 
review and approval. 

2.43.4.E Development of Summative Assessments 

2.43.4X1 Involvement of Committees 

The development of the summative assessments will involve several Indiana Committees, 
the IDOE, and the successful Respondent. Each of these committees has the mandate to 
use the Indiana Standards as the foundation of their work. To that end, it is not enough 
for them merely to align their work on the summative assessments with the Indiana 
Standards; the summative assessments must measure performance against the Indiana 
Standards. 
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The Respondent shall propose a process whereby all items for potential use on the 
summative assessments are reviewed and approved by the IDOE. The details of this 
process shall be finalized in collaboration between the successful Respondent and the 
IDOE. 

All items to be used on the summative assessments must be reviewed for alignment to 
Indiana Standards. The items must also be reviewed for fairness or bias to ensure that the 
summative assessments provide equitable measures for students with alternative cultural 
and ethnic backgrounds and diverse learning styles. 

These groups have already been established within the State of Indiana and their 
membership will be maintained or modified under the direction of the IDOE. 

2.43.4.E.2 Test Content and Item Format 

Hie ISTEP+ Info Center provides web links to sample and released items and scoring 
rubrics. Standards and other content-related information can be found at the Academic 
Standards site. These documents provide an indication of the types of skills and 
knowledge to be addressed within each content area for each grade level. Details of 
specifically what will be included in the summative assessments will be jointly 
determined by the successful Respondent and the IDOE. Items. will be a mix of multiple 
choice and constructed response and essay. The essential feature of all items included on 
the summative assessments is alignment to Indiana Standards and the measurement of 
performance relative to the Standards/Indicators. 

The Respondent is required to develop test items that align with Indiana's Academic 
Content Standards. There may be some Standards that will not be assessed through the 
summative assessments. These will be determined by the IDOE based on any needed 
modification of the current test blueprint. Annually, the existing item pool will be 
reviewed and the Respondent and the IDOE will agree to the proposed item development 
and piloting plan for the next testing year. The IDOE expects to receive high quality 
items with little need for revision and the IDOE reserves the right to reject at no cost any 
and all items that it deems as poor quality or not aligned to the Standards/Indicators. 
Furthermore, scoring rubrics that are of poor quality will also be rejected. Hie 
Respondent is responsible for ensuring that all items are carefully written, edited and 
proofed before being submitted to the IDOE for review. 

The Respondent shall submit with this RFP a chart indicating the number of items to be 
developed by Standard/Indicator for each subject (English/language arts, mathematics, 
science, and social studies) at each grade level (3 through 8). This chart will be subject to 
approval by the IDOE. The IDOE content personnel, along with the item review 
committees, will provide the Respondent with feedback regarding the match between test 
items that are developed and the Standards/Indicators. 

Beyond the contract period, the IDOE must be able to ensure the continuity of the 
Assessment system and efficiently move hard and electronic copies of the item banks 
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from one Respondent to another. The Respondent shall propose how this need may be 
met, including supporting rationale and costs. 

Hie Respondent shall also propose a strategy for allowing the IDOE to continue using 
Respondent-developed assessment items and the resulting data beyond the scope of any 
contract that results from this RFP. Royalty fees and other associated costs should be 
indicated in the proposal. 

2.43.4.E.3 Development of Rubrics for Constructed Response Items . . 

The development of rubrics for constructed response items will be the responsibility of 
the Respondent. All rubrics must be submitted along with the items to the IDOE for 
approval. 

The Respondent shall propose a strategy for incorporating their expertise into the rubric 
development process and shall propose a strategy for using the rubrics in scoring of 
constructed response items. 

The student's responses on the short answer or extended response items will be scored 
using the rubrics created for each item. While it would be desirable to have examples to 
guide the scoring, it is not required. The Respondent shall provide downloadable files 
of item-specific scoring rubrics. 

2.43.4.E.4 Item Development , 

The IDOE requires the Respondent to propose a detailed plan for item development The 
item development plan shall include: 

• Source of items; 

• Criteria that will be used to judge content validity and the technical 
quality of the items; 

• Criteria that will be used to develop test items across a wide range of 
depth of knowledge levels; 

• Criteria that will be used to judge the item quality; and 

• Procedures for ensuring that the above criteria will be used 
consistently throughout the item development process. 

It is expected that some of the items will be rejected in the process of the item reviews. 
Therefore, the number of items needed for the system refers to the number of items 
remaining after all item reviews have occurred. 
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2.4.3.4JE.5 Item Analysis 



The Respondent shall be responsible for providing appropriate item statistics and item 
analyses once an item has been piloted. Respondents are encouraged to propose analyses 
based on their experience and emerging statistical theory. The IDOE reserves the right to 
require trie Respondent to remove and replace any item with poor data. 

2.4 J.4.F Reporting 

Reporting fully, accurately, and in a timely manner is critical for any assessment 
program. ISTEP+ currently, produces a full range of traditional printed reports and labels 
at the student, classroom, school, and corporation level and supplies each corporation 
(and each charter and non-public school) with an electronic copy of their data. 

Respondents are invited to propose alternative reporting formats and methodologies that 
they believe would represent improvements. Specifically, Respondents are invited to 
propose reporting solutions that are not confined to paper reports, but rely on the Internet 
and vehicles other than paper reports to convey information about student achievement 
and school performance. Any such solution must maintain strict security and fully 
protect the individual rights of all students. In proposing possible alternative approaches 
to reporting, Respondents should be familiar with and should consider approaches that 
facilitate implementation of both the Indiana and federal accountability programs. 

2.43.4.G Pilot Testing 

All items used in ISTEP+ are currently pilot-tested, in stand alone forms, at the same 
time that ISTEP+ is administered. No items will be used in ISTEP+ tests without being 
piloted in Indiana and without having met acceptable standards of reliability and validity. 
All pilot samples must be statistically representative of Indiana students. Respondents 
must describe in detail the processes and procedures they recommend for pilot-testing all 
assessment items, explicitly relating their proposals to the new testing requirements 
discussed in this RFP for the administration of assessments. 

Respondents are invited to make their best proposal for administering pilot items in a 
manner that assures the reliability and validity of all items to be used in assessments, 
while minimizing to the extent possible the time required to pilot-test items. 

2.43.4.H Technical Analysis 

Respondents must ensure the reliability and validity of individual student scores. The 
technical analyses conducted by the Respondent for the summative assessment must meet 
nationally recognized professional and technical standards, as established by the 
Standards for Educational and Psychological Testing, published jointly by the American 
Educational Research Association, the American Psychological Association, and the 
National Council on Measurement in Education. All technical analyses and reports 
provided by the Respondent shall include appropriate cross-references to the relevant 
joint standards. 
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Hie Respondent shall include a section in the proposal showing evidence of experience 
identifying technical issues/problems and the subsequent solutions. This may include 
such things as scaling, equating, test construction, scoring drifts, etc. 

2.4J.4.H.1 Scaling and Equating 

For all elements of the summative assessment, the Respondent shall be responsible for 
proposing psychometric methods for scaling, equating, and linking test forms for each 
content area. Total test scores shall be equated and raw scores converted to scale scores. 
The Respondent must provide detail about the scale development that will be done for the 
summative assessment in the first year of operational administration and must provide 
rationales for the use of each of the scales, along with various psychometric issues and 
proposed solutions. 

The Respondent shall propose a detailed plan for building the necessary links for 
equating from form to form, specify procedures for estimating equating error across 
years, and propose a system for checking on scale drift The proposed equating 
methodology shall be clearly described in the Respondent's proposal. The Respondent is 
encouraged to propose alternative equating designs and time lines for establishing 
equated test forms. For each equating design proposed, the Respondent will provide 
advantages and disadvantages and make a recommendation for the use of one equating 
design over the other(s). 

2.43AR2 Validity 

Hie Respondent shall be responsible for establishing and documenting evidence of the 
validity of test scores from the summative assessment, which will be used as part of 
school and corporation accountability system. Evidence of validity shall include, but not 
necessarily be limited to, the following: 

a. Evidence that subject matter experts have determined that items and test 
forms represent an adequate sample of the content frameworks; 

b. Documentation of the rationale for the relative emphasis assigned to 
particular standards or indicators; 

c. Evidence that test item formats measure the intended content rather than 
some other construct; 

d. Evidence of the interrelationship among "standard" scores; 

e. Evidence that items were chosen on the basis of the test blueprint; and 

f. Evidence that alternate forms of each test cover the same content. 

2.43.4.HJ Reliability 

The Respondent will be responsible for establishing and documenting evidence of the 
reliability of test scores and scoring of hand-scorable items, including rater reliability. 
The assessments at all grade levels shall be developed so that evidence may be provided 
that scores are sufficiently valid and reliable at student levels for use in high-stakes 
decisions. Evidence of test score reliability shall include, but not necessarily be limited 
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to, inter-rater reliability, internal consistency of total scores and sub-scores, decision 
consistency, and standard error of measurement. Because of the focus on achievement 
levels, the most important evidence of reliability will be decision consistency 
information. 

The Respondent shall provide a plan for demonstrating evidence of score and scoring 
reliability for all hand-scorable item types. Hie Respondent shall provide evidence of 
consistency between raters. 

The Respondent shall propose a detailed plan for how it expects to complete all work 
associated with this task, including descriptions of designs of reliability studies, plans for 
conducting proposed studies, timelines, analyses to be conducted, statistics and reports to 
be provided, and the supporting rationale for the proposed design(s). This plan and 
associated work will be subject to negotiation and approval by the IDOE and/or the 
Testing Advisory Committee/Expert Panel. 

The Respondent must assist the IDOE with compiling additional evidence for 
subsequent peer reviews of any component of the summative assessment, as 
necessary, in compliance with federal requirements. 

2.43.4.1 Equating Studies 

The Respondent shall be responsible for conducting equating from form to form for all 
content areas. ISTEP+ is currently post-equated but Respondents are free to propose and 
defend what they believe is the most efficient and defensible equating methodology. Hie 
Respondent shall be responsible for the costs of an independent quality assurance check 
of the equating done for each administration of the summative assessment prior to the 
reporting of student results. 

2.43.4 J Standard Setting 

Under the supervision of the IDOE, the Respondent shall design and conduct standard 
setting studies, as appropriate and necessary, during the transition from ISTEP+ to the 
new summative assessments. Hie Respondent shall be responsible for all costs of - 
materials and staff support necessary to conduct the studies, complete appropriate 
analyses, and document the results in a comprehensive report. 

2.43.4.K Alignment Studies 

Under the supervision of the IDOE, the Respondent shall conduct alignment studies, as 
appropriate and necessary, after the transition from ISTEP+ to spring summative 
assessments. Alignment studies will be conducted for all content areas in all grade levels 
assessed. The Respondent shall be responsible for all costs of materials and staff support 
necessary to conduct such studies, complete appropriate analyses, and document the 
results in a comprehensive report. The IDOE may specify independent consultants to be 
used as independent facilitators for the alignment studies. The studies will be conducted 
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at a convenient, central location in Indiana and will include Indiana educators as the 
panel members. 

2.43.4X Technical Reports 

The Respondent shall prepare a technical report after each year's administration. The 
report should include statistical characteristics of assessment instruments (e.g., validity, 
reliability); design and results of scaling, linking, and standard setting; interpretations of 
assessment results; and other information as requested. The final report will be subject to 
approval by the IDOE. 

2.43 AM Comparability Studies 

At any time should any portion of the summative assessments be 
administered in more than one mode, it will be the responsibility of the 
Respondent to conduct and report a full comparability study. Such a study 
must determine the impact (if any) of different modes of administration and 
recommend to the IDOE any adjustments to scaling and/or equating that 
may be required. 

2.43.4.N Vendor Transition 

It is anticipated that some or all of the components of this RFP could be awarded to a 
Respondent other than the incumbent requiring that rights, property, processes, 
procedures and data be transferred from one vendor to another. All Respondents must 
present a detailed plan with a timeline for how they will smoothly, seamlessly and 
successfully carry out transitions between vendors. Any such plan must begin no later 
than 6 months prior to the first test administration that will be the responsibility of the 
Respondent. A complete "Transition Plan" will be due to the IDOE within 10 working 
days of the award of this Component of the RFP. 

2.43.4.0 Respondent Burden and Opportunity 

It is the burden of the Respondent to demonstrate whether and how its products and 
services meet NCLB and other legal requirements. The Respondent may utilize proof 
from other states and other sources, or other means, including working with IDOE, to 
meet this burden. 

2.43.5 Technical Proposal 

The IDOE seeks proposals from Respondents regarding the development (including item 
development and content review), administration (online and paper-and-pencil), standard 
setting, scoring, research and review (comparability studies, rescore study) and collateral 
activities (online scheduling and registration, communication and reporting) of end-of- 
year summative assessments in the following subject matter areas in grades 3 through 8: 
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• Annual summative assessments in grades 3-8, which contain progress and writing, 
in the areas of English/language arts and mathematics. 

AND 

• Annual summative assessments, which contain progress and writing, in grades 3-8 
for science, and in grades 4, 6, and 8 for social studies. 

OR 

• Summative assessments, which contain progress and writing, in grades 5 and 7 for 
science, and in grades 4, 6, and 8 for social studies. 

Continued development and use of paper- and -pencil tests for some schools is a practical 
necessity, but this RFP seeks proposals for the online format in the summative 
assessments. " 

The IDOE anticipates entering into a multi-year agreement with the Respondent for ' 
summative assessments beginning in the 2008-2009 school year, with the option to 
extend for additional multi-year periods. 

The IDOE requires Respondents to propose a detailed work plan that accomplishes the 
stated objectives. Refining this work plan with the IDOE will be an obligation of the 
successful Respondent, and the revised work plan will be incorporated into the final - 
contract documents with the Respondent. 

The work plan should permit the Respondent to conduct the necessary preliminary 
activities involved in preparation for the summative assessments. The IDOE seeks a 
workable long-term approach to the summative assessments consistent with the overall 
assessment system parameters in the Plan. The IDOE has provided some specificity 
about its needs, and the technical requirements identified above must be satisfied, but 
Respondents are welcome to make suggestions and suggest alternative strategies or 
approaches (including those already in use elsewhere) that could lead to a more effective 
and efficient assessment process. 

The Respondent must identify whether and, if so, how its proposed summative 
assessments correlate to national and international standards and other indicators of 
student performance and whether schools and the IDOE will be able to utilize the results 
of the summative assessments to benchmark student results in that context 

The Respondent must demonstrate how its proposed summative assessments will: 

• Be valid and reliable; 

• Reflect and align to Indiana's curricular aims and Standards and predict to success in 
both preparation for higher education and for career and technical pursuits; 

• Conform to the requirements of Indiana and federal law, including NCLB; 

• Be tracked by the STN Indiana assigns to the student; 

• Enable teachers and schools to monitor individual and collective student results 
demographically across student populations and longitudinally across time; 
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• Give individual teachers feedback that is performance-based and may be used to 
guide instruction and student progress, and enable teachers and schools to evaluate 
and adjust instruction to improve student achievement; 

• Inform parents as to the academic progress of their students; 

• Enable the results of the assessments to be included in individual student learning 
portfolios and in longitudinal studies of student performance within schools and 
across the state. 

2.43.5.A. Operational Administration of Stimulative Assessments 

Summative assessment items that have not already been written and pilot-tested and 
operationally deployed must be. The IDOE requires the Respondent to provide all 
additional services necessary to administer these assessments in a production or 
operational environment. 

General 

1 . The IDOE invites proposals for effective and efficient administration and grading of 
the writing assessments, recognizing that the assessment should use essay and other 
open-ended questions that measure student use of grammar and composition for 
effective writing, higher-order thinking, and applied skills, and that it should be able 
to be administered and scored in a timely manner, enabling common reporting with 
the progress assessment at the end of the school year. 

2. The IDOE invites proposals for effective and efficient administration and scoring of 
the progress assessments, recognizing that the assessments must be concise in time to 
take; in the context of adequate technical rigor and quality necessary to meet federal 
or other applicable standards; should have no open-ended or essay questions and 
should use other means to assess higher-order thinking skills and curricular aims; 
should be administered near the end of the school year; and scored in a timely 
manner, enabling common reporting with the writing assessment at the end of the 
school year. 

3. The summative assessments administered by the Respondent must be available to all 
students, and the Respondent must propose a plan for appropriate accommodations to 
all students. 

4. Although both testing formats will be available to schools during the first multi-year 
agreement, schools will be required to select either the electronically enabled format 
or the paper-and-pencil format for each content area. Paper-and-pencil format for the 
progress assessment must be able to be electronically scored to support the necessary 
turn-around time for reporting by the end of the school year. 

5. The IDOE places great importance on test confidentiality, integrity and security. The 
Respondent must propose specific security measures considered appropriate for 
operational administration of the summative assessments. 
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6. The Respondent must assume access to and use of existing summative assessment 
forms, item banks and related material where they have been developed in previous 
pilot and operational tests. These materials are the sole property of the State of 
Indiana, and the Respondent will be granted appropriate license and access to these 
materials. The Respondent must include a plan for transition of items, test forms, and 
student responses. All new materials developed as a result of this RFP will be the 
sole property of the State of Indiana. Where the Respondent proposes to utilize 
forms, item banks or other materials developed elsewhere, the Respondent must grant 
appropriate and perpetual license and access to such materials. 

Electronically-Enabled Administration 

1 . The Respondent must be responsible for all operational and support tasks associated 
with administering the electronically-enabled summative assessments, including: 

a. Design and development of the annual test administration cycle and schedule; 

b. Development and publication of web and print instructions, manuals, and other 
communications to schools, including web trainings and online tutorials; 

c. Publication of tests and scoring guides; , ( 

d. Development of an online scheduling system for schools to select test windows 
and delivery formats that will include both testing formats; 

e. Design of a mandatory program to assess the technology capabilities of schools 
and respond to various operating systems and browsers; 

f. Development of practice tests (an online set of items that allow students to 
become familiar with online testing and any tools needed by students) and general 
test preparation materials (including an item sampler that provides examples of 
the item types with content that students will see on the test); 

g. Scoring/grading of completed online tests (the fee schedule allows for pricing of 
human scoring or artificial intelligence or other scoring of essay and open-ended 
items on the writing assessment); 

h. Development of anchor sets and training sets for essay and open-ended items for 
the writing assessment; 

i. Development of plans for providing accommodations; 

j. Provide appropriate security arrangements for test content and test administration; 
. k. Preparation and distribution of online reports for both testing formats to teachers, 

school officials, corporation officials and the IDOE, with appropriate security 

arrangements to safeguard access to reports; the Respondent will need to 

incorporate the cut scores into these reports; 
1. 1 Development of promotional materials for use with schools; 
m. Distribution of electronic communication necessary for registration, testing, 

reporting, and general dissemination of summative assessment information; 
n. Development of a plan for re-scoring essays in cases where schools and the 

Respondent disagree; 
o. Provision of a technical report that includes information regarding administration, 

psychometrics, and summary data for the summative assessments. 

2. Hie Respondent should assume and plan for new and secure item rotation and testing 
in each successive year's administration of the summative assessments. The 
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Respondent should provide a separate report and analysis to the IDOE on new items 
introduced in each test administration; this report should include item data analysis 
and any recommendations for improvement of the items. 

3. The Respondent must deliver to the IDOE aggregate score reports and incorporate the 
cut-scores into student, classroom, school, corporation and state-level reports of 
operational test results. The Respondent must deliver a system in which all score 
reports, both online and paper-and-pencil, are delivered in one format to schools, such 
that scores are compiled and shared with schools in the same manner, regardless of 
the testing format selected. The IDOE prefers an online system that generates printer- 
friendly reports at the student, classroom, school, corporation, and state levels. 

4. All functions of the online system must be platform, operating system, and browser 
independent. All functions of the online system must work in recent versions of 
Netscape Navigator, Internet Explorer, Firefox, Mozilla, Safari, and Linux, or the 
Respondent must present an alternate plan to address the various platforms, operating 
systems, and browsers used by schools in a secure testing environment. 

5. The Respondent must design and implement a Scoring Reliability (auditing) system 
for the scoring of essay and open-ended items and budget for a reliability system that 
includes a representative sample of student responses. 

Paper-and-Pencil Administration 

1 . The tests administered by the Respondent must be available to all students, and the 
Respondent must propose a plan for appropriate accommodations to all students. 
Therefore, a large print version of the test will be necessary as well as the possibility 
of a Braille version. 

2. The IDOE requests proposals from the Respondent for scoring of the paper-and- 
pencil tests, including the open-ended and essay items. The Respondent may suggest 
scoring of the paper-and-pencil-based writing assessments by electronic means or by 
human means; scoring of the paper-and-pencil-based progress assessments must be 
by electronic means. In addition to providing scoring guides, this will require that 
schools be able to submit exam results directly to the Respondent for purposes of 
consolidated score reporting at the student, school, corporation, and state levels. If 
human scoring of open-ended and essays is proposed, the Respondent must propose a 
system for scorer training at the school level, including the administration and scoring 
of tests. 

3. The Respondent must design and implement a Scoring Reliability (auditing) system 
for the scoring of open-ended and essay items and budget for a reliability system that 
includes a representative sample of student responses. 

4. The Respondent will be responsible for all operational and support tasks associated 
with administering the paper-and-pencil version of the summative assessments, 
including: 
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a. Design and development of the annual test administration cycle and schedule; 

b. Development and publication of web and print instructions, manuals, and other 
communications to schools, including web trainings and online tutorials; 

c. Publication of tests and scoring guides; 

d. Development of an online scheduling system for schools to select test windows 
and delivery formats; 

e. Development and execution of a plan for the ordering, delivery, and pick-up of 
paper- and-pencil tests and materials; 

f. Development and publication of practice tests and general test preparation tools; 

g. Development of training materials for paper-and-pencil scoring of open-ended 
and essay items; 

h. Scoring/grading of the essays and open-ended questions on the writing assessment 
and the questions on the progress assessment, as well as a scoring reliability check 
(audit) of teacher-scored tests if not human-scored by the Respondent; 

. i. Development of plans for providing accommodations; ' . 
j. ' Provide appropriate security arrangements for test content and test administration; 
k. Preparation and distribution of online reports to teachers, school officials, 
corporation officials and the IDOE, with appropriate security arrangements to 
safeguard access to reports; the Respondent will need to incorporate the cut scores 
into these reports; 

1. Development of promotional materials (electronic and paper) for use with 
schools. 

5. The Respondent must assume and plan for new item rotation and testing in each 
successive year's administration of each summative assessment. The Respondent 
must provide a separate report to the IDOE on new items introduced in each test 
administration; this report must include item data analysis and recommendations, if 
any, for improvement of the items. 

6. The Respondent must deliver to the IDOE aggregate score reports and incorporate the 
cut scores into, student, classroom, school, corporation and state-level reports of 

. operational test results. The Respondent must deliver a system in which all score 
' reports, both online and paper-and-pencil, are delivered in one format to schools, such 
that scores are compiled and shared with schools in the same manner, regardless of 
the testing format selected. The IDOE prefers an online system that generates printer- 
friendly reports at the student, classroom, school, corporation and state levels. 

Operational Performance Standards for Acceptable Administration - All Testing 

• Aggregate school or class performance must be reported by the end of the school 
year. 

• Aggregate state-level reporting to the IDOE must be reported by June 30 for each 
academic year. 

Computer-Based Testing 

• Students using online testing arrangements must achieve response times meeting 
current industry standards. 
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• School officials must receive student test results within 24 hours of test completion. 



Paper-and-Pencil Testing 

• Students using paper-and-pencil tests must receive score reports within 7 calendar 
' days, starting from receipt of the test by the Respondent. 

2.43.5.B. Online Scheduling and Registration, Communication, and Reporting 

System 

The Respondent must provide a system in which: 

1 . Corporations can complete online registration for online and paper-and-pencil 
assessments. 

2. The IDOE will have a delivery method for necessary electronic and paper 
communications with corporation and school officials for online and paper-and-pencil 
tests, separately and combined. 

3. An online reporting system for student, school, corporation and state-level data, as 
described in Section A, is available for schools, corporations and the State. 

4. A consolidated help desk will exist to assist corporations with all facets of technology 
issues related to online testing, registration, and reporting, as well as paper-and-pencil 
logistical issues. 

2.43.5.C. Item Development and Content Review 

1 . New items will need to be developed for new operational assessments. Additional 
items will need to be developed periodically. Examples of item types that are 
desirable in layout and format for online and paper-and-pencil items can be found at: 
http://doe.state.in.us/istep/publications.html . Exact quantities and nature of items 

. appropriate to each summative assessment - multiple-choice, open-ended, essay, 
numeric response and equation/expression, graphing, constructed response or other - 
will be determined by the IDOE. 

2. The Respondent will be responsible for conducting passage reviews, content reviews 
and sensitivity/bias reviews, including setting up meetings and facilitating the reviews 
for new summative assessments. 

3. Test form construction will be based on item analysis. The IDOE anticipates 
additional test forms may be constructed based on field-testing of new items. 
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2.43.5 J). Standards (Cat Score) Setting 



The IDOE expects that Standards (cut score) Setting will occur for each summative 
assessment. The Respondent must provide the structure, agenda, support and materials 
for the Standards Setting. The IDOE will manage the logistics and recruit panelists. 

2.43.5.E. Comparability Studies 

The IDOE envisions that a comparability study will be completed as each operational 
administration of a summative assessment concludes. 

2.43.5.F. Rescore Study and Report 

A restoring of open-ended and essay items for online and paper-and-pencil assessments 
must be completed. The Respondent must define rescore samples, perform rescore 
studies, and generate reports of representative samples for all operational assessments. 

2.43.6 Cost Proposal 

For purposes of planning and budget preparation, the IDOE expects the number of tests 
administered to vary across schools. For purposes of the Respondent's cost proposal, the 
Respondent should plan for and assume the quantities designated on the formatted cost 
sheet (fee schedule), Attachment E. The IDOE prefers a cost proposal that includes a 
cost structure based on the number of assessments actually administered. 
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2.4.4 Component 4: High School Core 40 End-of-Course Assessments 



2.4.4.1 Background Information 

Aligned with Indiana's Academic Standards, the Core 40 End-of-Course 
Assessment System is a set of end-of-course tests measuring what students know 
and are able to do after completing specific Core 40 courses. Established as part 
of Indiana's school accountability system under Public Law 221-1999 (P.L. 221), 
the Core 40 End-of-Course Assessments ("ECAs") are designed to ensure the 
quality, consistency and rigor of Core 40 courses across the state. 

The Core 40 End-of-Course Assessment System (ECA) is a key component of Indiana's 
P-16 Plan for Improving Student Achievement and the Assessment System adopted by 
the ISBE and IDOE. 

As part of the P-16 Plan, the State is currently working with Indiana's colleges and 
universities to use the End-of-Course Assessments for dual purposes: as part of the high 
school competency assurance and as college placement exams for seamless. transition 
from high school to college. The State's ECA System currently includes administering 
End-of-Course Assessments in Algebra I, Algebra n. Biology I, English 1 1, and U.S. 
History. This RFP is intended to address the State's longer-term needs for EC As for all 
primary Core 40 courses. 

2.4.4.2 History 

Indiana's history with end-of-course assessments is as follows: 

• Beginning with the 2003-04 school year, the Algebra I and English 1 1 assessments 
became operational under the state's accountability system for schools. In order for 
schools to be considered for the top two performance categories under P.L. 221, 
schools are required to participate in the Operational Administrations of the EC As. 

• In 2002, schools participated in the first ECA pilot, which included online and paper- 
and- pencil administration in English/language arts and paper-and-pencil testing in 
Algebra I. 

• In 2003, all schools with students enrolled in Algebra I or English 1 1 had the 
opportunity to participate in the online pilot. 

• Spring 2004 marked the first operational administration of the Algebra I and English 
1 1 assessments. Beginning in 2004, schools again had a choice between online or 
paper-and-pencil administration for the operational assessments. 

• In winter 2005, Algebra I and English 1 1 were offered as operational online 
assessments. In spring 2005, three content areas were piloted: Algebra II and Biology 
I were online pilots, while U.S. History was a paper-and-pencil pilot Algebra I and 
English 1 1 were offered as operational online or paper-and-pencil assessments. 

• In winter 2006, Algebra I and English 1 1 were offered as operational online 
assessments. In spring 2006, Algebra II and Biology I were once again online pilot 



Exhibit D, page 61 of 92 



tests, and U.S. History was once again a paper-and-pencil pilot test. Algebra I and - 
English 1 1 were offered as operational online or paper-and-pencil assessments. 
• In spring 2007, Algebra II and Biology I assessments will become operational in both 
online and paper-and-pencil formats. The U.S. History assessment will become 
operational in paper-and-pencil format. Algebra I and English 1 1 assessments will 
continue to be offered operationally in both online and paper-and-pencil formats. 

2.4.43 Elements 

As part of the Assessment System, the End-of-Course Assessments will be given in each 
of the basic Core 40 course areas (English, mathematics, science, and social studies). 
They will reflect both the rigor of the state's Academic Standards and preparation for 
post-secondary study in and work using such subjects. Pursuant to the dual purpose of 
their use, the assessments will be of such rigor as to enable institutions of higher 
education to use them for student placement. 

The assessments will be given at the end of the course taken, whether it is during the 
spring or fall semester. All students taking the course will take the assessment Students 
not in the course but whose other experience indicates the appropriateness of taking the 
assessment may do so. Where students take an Advanced Placement version of a course 
that has an End-of-Course Assessment, they will not have to take the End-of-Course 
Assessment, and the AP test result will be noted in its stead. Students will be able to bank 
ECA scores taken prior, to high school. 

The format of the assessments must enable electronic administration and scoring and 
provide compatibility to move to online assessments when such capacity exists. The 
results of the assessment will be reported to the student's parents, be part of the student's 
transcript, and be included in the student's learning portfolio. The results of the 
assessments will be used to confirm student difficulty with, proficiency in, or mastery of 
the subject matter of the course and thus to assist in student course selection, counseling 
and career planning (and, if needed, remediation) through high school. 

End-of-Course Assessments will be tracked by an individual student's STN, so that the 
results will be both portable over geography (as the student moves from school to school 
or corporation to corporation) and available over time (as the student rises through grades 
K-12 to graduation and ultimately into post-secondary education). They will become part 
of each student's learning portfolio, to evidence the student's progress to proficiency in 
the auricular aims. ECA results will be reported by school and corporation. 

Local decision will determine whether the assessment is used in whole or part to 
determine the student's grade in the course. Over time, student success on these 
assessments will be recognized and rewarded by the State. 

Eventually, all students will take End-of-Course Assessments in English, mathematics, 
science, and social studies courses; their scores will be reported on their transcripts and 
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used to qualify for diploma endorsements, with the expectation that over time the results 
will be used in awarding Core 40 diplomas. 

End-of-Course Assessments in Algebra I, English 10, and Biology I will be required of 
all students to meet current NCLB requirements for high school course assessments. 
Should NCLB requirements for assessments in high school be expanded, additional 
ECAs will be used for that purpose. 

The State's use of the GQE will be replaced by student success in End-of-Course 
Assessments in Algebra I and English 10. Students will have the same right to retake 
these assessments if they do not pass them as currently exists under the GQE. 

In appropriate courses or areas (for example, government and financial literacy), ECAs 
may be provided for local use to support confirmation of student proficiency in such 
areas. 

Further ECAs may be developed over time for courses in grades 1 1 and 1 2 that have 
targeted use for such an assessment They may be provided, for example, in courses 
relevant to career and technical education subject areas; to enable a student to 
demonstrate specific proficiency adequate for placement in post-secondary study; or for 
confirmation of ability to enter specific post-secondary work. Use of these targeted 
assessments will be voluntary at the local level. 

Local schools, corporations, and communities will continue to own the development of 
their students 1 written expression through their high school years. Course assignments 
and examinations during the year, project-based learning and other locally-determined 
curricular contexts will provide ample opportunity for assessment of each student's 
thinking and writing. 

2.4.4.4 Technical Requirements 

Respondents must acknowledge their understanding and acceptance of the listed technical 
requirements. Respondents must also provide a narrative for each requirement that 
demonstrates its ability to meet the stated requirement 

2.4.4.4 A Current GQE 

The current grade 10 ISTEP+ assessment required for graduation is commonly known as 
the Graduation Qualifying Exam (GQE). The GQE categorizes students as above passing 
or below passing separately in English/language arts and mathematics, based on their 
scores in relation to cut scores established by the Indiana State Board of Education. Each 
student's composite scores for each subject are based on items from both the selected 
response and constructed response components of ISTEP+. 

This test is a credentialing test, by law a mandatory part of the graduation requirements 
and as such, subject to legal review. The current test and its successors must meet all of 
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the requirements of "opportunity to learn." Timelines and project plans must take into 
account the necessary retest opportunities, any needed "grandfathering" as a transition is 
made from one credentialing test to another, and all other necessary work to ensure the 
successful legal defensibility of a "graduation" test. 

In addition to the current fall administration of ISTEP+, including the GQE, a GQE retest 
is administered in the spring (March) to students who have not yet met Indiana Academic 
Standards. 

A student scoring below state standards in the fall of 2007, for example, is entitled to be 
retested for the first time in the fall of 2008, and during each subsequent fall and/or 
spring test administration: The student is retested only in the subject area in which he/she 
did not meet standards (i.e., if a student met standards in mathematics but did not meet 
standards in English/language arts, the student is retested only in English/language arts). 

Students who retake the GQE in the fall are required to test on the same three days as the 
grade 10 students who are being administered the Graduation Qualifying Exam for the 
first time. All students being administered the GQE in a spring retest are required to take 
the test(s) during the established three-day testing window. Make-up testing for the GQE 
is not permitted. 

Students who have left high school without a diploma are also eligible to participate in 
GQE testing. The successful Respondent must provide for the administration of the GQE 
and successors to adults, as well as to high school students. 

Indiana wishes to move from using traditional achievement tests (GQE) for the high 
school graduation test and for NCLB accountability to the use of three specific End of 
Course Assessments. English 10, Algebra I and Biology would become high stakes 
accountability tests with English 10 and Algebra I becoming the "new" graduation test. 
These 3 ECAs by definition should be available when students complete the course, not 
at specific times once or twice a year. This need will bring additional complications to 
the use of these 3 ECAs as both graduation test (with the attendant retest requirements) 
and as an NCLB accountability test. Respondents must directly address these added 
complexities which may require additional forms, increased security and greater 
psychometric complexity than for the ECAs not a part of the NCLB and graduation 
requirements. 

2.4.4.4.B Differentiation among ECAs 

Respondents must clearly distinguish between the products, services and functions that 
will apply to each ECA. All the ECAs have the potential to be "high stakes" 
assessments, but at the outset they fall into three categories: 

a) ECAs for both graduation and NCLB - English 10 and Algebra L 

b) ECA for NCLB - Biology I, and 

c) All other requested ECAs. 
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2.4.4.4.C Technical Requirements for NCLB ECAs 



The following technical requirements will apply to a) and b) [ECAs for both graduation 
and NCLB - English 10 and Algebra 1, and ECA for NCLB - Biology I]. They will not 
apply to c) [All other requested ECAs], 

2.4.4.4.C.1 Program Manager and Project Management Team 

The Respondent shall assign a Program Manager, dedicated full-time to this project, and 
assemble a project management team to oversee and coordinate the efforts of the 
contractor and all related subcontractors. The Program Manager shall serve as the 
primary liaison with the IDOE for all components of the project. The Program Manager 
must have demonstrated previous experience with managing a large, customized 
assessment project similar in scope and nature to the ECA assessments. 

A cohesive, dedicated, skilled core team is critical to the ECA assessment project , At a 
minimum, this core team of key personnel shall be listed in the Respondent's proposal. 
Personnel commitments and identifying core team key personnel as specific individuals 
are a mandatory requirement for the work to be performed under this contract and subject 
to approval by the IDOE Division of School Assessment Hie Respondent shall provide 
an organizational chart showing all key staff and offices assigned to work on the various 
aspects of the ECA assessments. Roles and responsibilities for all key staff shall be 
identified. 

Except in the case of illness, death, or leave of absence and so long as the personnel 
remain partners, principals, or employees of Respondent no re-deployment of any 
member of the core team as required by the work plan may be made without prior written 
consent of the IDOE Division of School Assessment, which shall not be withheld without 
good cause. Replacement of such personnel, if approved, shall be with personnel of 
equal or greater ability and qualifications. 

2.4.4.4.C.2 Project Plans and Schedules 

The Respondent shall continuously monitor the ongoing operations of the ECA 
assessments using a detailed project plan and schedule. The Respondent shall provide an 
annual updated project plan and schedule for joint use by the IDOE and the Respondent 
in monitoring all program activities. The IDOE desires appropriate direct real time 
access to the project schedule. 

2.4.4.4.C3 Status and P lanning Meetings 

The Respondent shall participate in weekly status meetings with the IDOE. These 
meetings will be conducted by telephone and/or by pre-arranged videoconferencing, 
Web-ex, or via other communication technology as proposed by the Respondent The 
Respondent shall be responsible for providing written documentation of the weekly status 
meetings within 24 hours of each meeting. The Respondent shall also provide weekly 
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written status reports that shall include but not be limited to, reports on the status of 
ongoing activities, decisions made, decisions pending, activities completed, problems, 
and timelines for scheduled activities. Weekly status reports shall be in a format that is 
approved by the IDOE Division of School Assessment. 

The Respondent shall participate in and support all associated costs for a minimum of six 
planning meetings of the IDOE and Respondent project management teams, other 
contractor management staff responsible for services provided to the IDOE, and 
representatives of the Respondent's senior management team, as necessary. Three two- 
day meetings will be held at the Respondent's site and three two-day meetings will be 
held in the State of Indiana at a site convenient to the IDOE staff (a total of 12 meeting 
days). The Respondent shall be responsible for all costs associated with these meetings. 
The IDOE staff attending the meetings will include the assistant superintendent, the 
assessment director, and up to four education associates. During the initial "start-up" 
phase of this project there may be a need for additional meetings; these will be at the 
Respondent's expense with dates, times and locations to be mutually agreed upon by the 
Project Manager and the Director of School Assessment The Respondent shall submit 
the meeting notes to the IDOE within five (S) working days of each meeting for final 
review and approval. 

2.4.4.4.C.4 Involvement of Committees 

Hie development of the ECA assessments to which the NCLB applies will involve 
several Indiana Committees, the IDOE, and the successful Respondent. Each of these 
committees has the mandate to use the Indiana Standards as the foundation of their work. 
To that end, it is not enough for them merely to align their work on the ECA assessments 
with the Indiana Standards; the ECA assessments must measure performance against the 
Indiana Standards. 

The Respondent shall propose a process whereby all items for potential use on the ECA 
assessments are reviewed and approved by the IDOE. The details of this process shall be 
finalized in collaboration between the successful Respondent and the IDOE. 

All items to be used on the ECA assessments must be reviewed for alignment to Indiana 
Standards. The items must also be reviewed for fairness or bias to ensure that the ECA 
assessments provide equitable measures for students with alternative cultural and ethnic 
backgrounds and diverse learning styles. 

These groups have already been established within the State of Indiana and their 
membership will be maintained or modified under the direction of the IDOE. 

2.4.4.4.G5 Test Content and Item Format 

The ISTEP+ Info Center provides web links to sample and released items and scoring 
rubrics. Standards and other content-related information can be found at the Academic 
Standards site. These documents provide an indication of the types of skills and 
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knowledge to be addressed within each content area for each grade level. Details of 
specifically what will be included in the ECA assessments will be jointly determined by 
the successful Respondent and the IDOE. Items will be a mix of multiple choice and 
constructed response, including short answer, extended response, and essay. Hie 
essential feature of all items included on the ECA assessments is alignment to Indiana 
Standards and the measurement of performance relative to the Standards/Indicators. 

The Respondent is required to develop test items that align with Indiana's Academic 
Content Standards. There may be some Standards that will not be assessed through the 
ECA assessments. These will be determined by the IDOE based on any needed 
modification of the current test blueprint. Annually, the existing item pool wilt be 
reviewed and the Respondent and the IDOE will agree to the proposed item development 
and piloting plan for the next testing year. Hie IDOE expects to receive high quality 
items with little need for revision and the IDOE reserves the right to reject at no cost any 
and all items that it deems as poor quality or not aligned to the Standards/Indicators. 
Furthermore, scoring rubrics that are of poor quality will also be rejected. The 
Respondent is responsible for ensuring that all items are carefully written, edited and 
proofed before being submitted to the IDOE for review. 

Hie Respondent shall submit with this RFP a chart indicating the number of items to be 
developed by Standard/Indicator for each subject (English/language arts, mathematics, 
science, and social studies) at each grade level. This chart will be subject to approval by 
the IDOE. The IDOE content personnel, along with the item review committees, will 
provide the Respondent with feedback regarding the match between test items that are 
developed and the Standards/Indicators. 

Beyond the contract period, the IDOE must be able to ensure the continuity of the 
Assessment system and efficiently move hard and electronic copies of the item banks 
from one Respondent to another. The Respondent shall propose how this need may be 
met, including supporting rationale and costs. 

The Respondent shall also propose a strategy for allowing the IDOE to continue using 
Respondent-developed assessment items and the resulting data beyond the scope of any 
contract that results from this RFP. Royalty fees and other associated costs should be 
indicated in the proposal. 

. 2.4.4.4.C.6 Development of Rubrics for Constructed Response Items 

The development of rubrics for constructed response, both short answer and extended 
response items, will be the responsibility of the Respondent. All rubrics must be 
submitted along with the items to the IDOE for approval. 

Hie Respondent shall propose a strategy for incorporating their expertise into the rubric 
development process and shall propose a strategy for using the rubrics to select anchor 
papers to assist Indiana educators in local scoring of constructed response items. 
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Hie student's responses on the short answer or extended response items will be scored 
using the rubrics created for each item. 

2.4.4.4.C.7 Item Development 

The IDOE requires the Respondent to propose a detailed plan for item development The 
item development plan shall include: 

• Source of items; 

• Criteria that will be used to judge content validity and the technical 
quality of the items; 

• Criteria that will be used to develop test items across a wide range of 
depth of knowledge levels; 

• Criteria that will be used to judge the item quality; and 

• Procedures for ensuring that the above criteria will be used 
consistendy throughout the item development process. 

It is expected that some of the items will be rejected in the process of the item reviews. 
Therefore, the number of items needed for the system refers to the number of items 
remaining after all item reviews have occurred. 

At each grade level and in each content area, at least 35% of the overall items shall be 
short answer or extended response items. 

2.4.4.4.C.8 Item Analysis 

The Respondent shall be responsible for providing appropriate item statistics and item 
analyses once an item has been piloted. Respondents are encouraged to propose analyses 
based on their experience and emerging statistical theory. The IDOE reserves the right to 
require the Respondent to remove and replace any item with poor data. 

2.4.4.4.C.9 Reporting 

Reporting fully, accurately, and in a timely manner is critical for any assessment 
program. ISTEP+ currently produces a full range of traditional printed reports and labels 
at the student, classroom, school, and corporation level and supplies each corporation 
(and each charter and non-public school) with an electronic copy of their data. 

Respondents are invited to propose alternative reporting formats and methodologies that 
they believe would represent improvements. Specifically, Respondents are invited to 
propose reporting solutions that are not confined to paper reports, but rely on the Internet 
and vehicles other than paper reports to convey information about student achievement 
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and school performance. Any such solution must maintain strict security and fully 
protect the individual rights of all students. In proposing possible alternative approaches 
to reporting, Respondents should be familiar with and should consider approaches that 
facilitate implementation of both the Indiana and federal accountability programs. 

2.4.4.4.C.10 Pilot Testing. 

All items used in ISTEP+ are currently pilot-tested, in stand alone forms, at the same 
time that ISTEP+ is administered. No items will be used in ISTEP+ tests without being 
piloted in Indiana and without having met acceptable standards of reliability and validity. 
All pilot samples must be statistically representative of Indiana students. Respondents 
must describe in detail the processes and procedures they recommend for pilot-testing all 
assessment items, explicitly relating their proposals to the new testing requirements 
discussed in this RFP for the administration of assessments. 

Respondents are invited to make their best proposal for administering pilot items in a 
manner that assures the reliability and validity of all items to be used in assessments, 
while minimizing to the extent possible the time required to pilot-test items. 

2.4.4.4.C.11 Technical Analysis 

Respondents must ensure the reliability and validity of individual student scores. The 
technical analyses conducted by the Respondent for the ECA assessment must meet 
nationally recognized professional and technical standards, as established by the 
Standards for Educational and Psychological Testing, published jointly by the American 
Educational Research Association, the American Psychological Association, and the 
National Council on Measurement in Education. All technical analyses and reports 
provided by the Respondent shall include appropriate cross-references to the relevant 
joint standards. 

Hie Respondent shall include a section in the proposal showing evidence of experience 
identifying technical issues/problems and the subsequent solutions. This may include 
such things as scaling, equating, test construction, scoring drifts, etc. 

2.4.4AC.12 Scaling and Equating 

For all elements of the ECA assessment, the Respondent shall be responsible for proposing 
psychometric methods for scaling, equating, and linking test forms for each content area. 
Total test scores shall be equated and raw scores converted to scale scores. Hie Respondent 
must provide detail about the scale development that will be done for the ECA assessment in 
the first year of operational administration and must provide rationales for the use of each of 
the scales, along with various psychometric issues and proposed solutions. 

The Respondent shall propose a detailed plan for building the necessary links for 
equating from form to form, specify procedures for estimating equating error across 
years, and propose a system for checking on scale drift. The proposed equating 



Exhibit D, page 69 of 92 



methodology shall be clearly described in the Respondent's proposal. Hie Respondent is 
encouraged to propose alternative equating designs and time lines for establishing 
equated test forms. For each equating design proposed, the Respondent will provide 
advantages and disadvantages and make a recommendation for the use of one equating 
design over the other(s). 

2A4.4.C13 Validity 

The Respondent shall be responsible for establishing and documenting evidence of the 
validity of test scores from the ECA assessment, which will be used as part of a school and 
corporation accountability system. Evidence of validity shall include, but not necessarily be 
limited to, the following: 

a. Evidence that subject matter experts have determined that items and test . . 
forms represent an adequate sample of the content frameworks; 

b. Documentation of the rationale for the relative emphasis assigned to 
. particular standards or indicators; 

c. Evidence that test item formats measure the intended content rather than 
some other construct; 

d. Evidence of the interrelationship among "standard" scores; 

e. Evidence that items were chosen on the basis of the test blueprint; and 

f. Evidence that alternate forms of each test cover the same content. 

Z4.4.4.C14 Reliability 

The Respondent will be responsible for establishing and documenting evidence of the 
reliability of test scores and scoring of hand-scorable items, including rater reliability. The 
assessments at all grade levels shall be developed so that evidence may be provided that 
scores are sufficiently valid and reliable at student levels for use in high-stakes decisions. 
Evidence of test score reliability shall include, but not necessarily be limited to, inter-rater 
reliability, internal consistency of total scores and sub-scores, decision consistency, and 
standard error of measurement Because of the focus on achievement levels, the most 
important evidence of reliability will be decision consistency information. 

The Respondent shall provide a plan for demonstrating evidence of score and scoring 
reliability for all hand-scorable item types. The Respondent shall provide evidence of 
consistency between raters. 

The Respondent shall propose a detailed plan for how it expects to complete all work 
associated with this task, including descriptions of designs of reliability studies, plans for 
conducting proposed studies, timelines, analyses to be conducted, statistics and reports to 
be provided, and the supporting rationale for the proposed design(s). This plan and 
associated work will be subject to negotiation and approval by the IDOE and/or the 
Testing Advisory Committee/Expert Panel. 

The Respondent must assist the IDOE with compiling additional evidence for 
subsequent peer reviews of any component of the ECA assessment, as necessary, in 
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compliance with federal requirements. 

2.4.4.4.C.15 Equating Studies 

The Respondent shall be responsible for conducting equating from form to form for all 
content areas. ISTEF+ is currently post-equated but Respondents are free to propose and 
defend what they believe is the most efficient and defensible equating methodology. The 
Respondent shall be responsible for the costs of an independent quality assurance check 
of the equating done for each administration of the ECA assessment prior to the reporting 
of student results. 

2.4.4.4.C.16 Standard Setting 

Under the supervision of the 1DOE, the Respondent shall design and conduct standard 
setting studies, as appropriate and necessary, during the transition from ISTEP+ to the 
new ECA assessments. The Respondent shall be responsible for all costs of materials and 
staff support necessary to conduct the studies, complete appropriate analyses, and 
document the results in a comprehensive report. 

2.4.4.4.C.17 Alignment Studies 

Under the supervision of the IDOE, the Respondent shall conduct alignment studies, as 
appropriate and necessary, after the transition from ISTEP+ to spring ECA assessments. 
Alignment studies will be conducted for all content areas in all grade levels assessed. 
The Respondent shall be responsible for all costs of materials and staff support necessary 
to conduct such studies, complete appropriate analyses, and document the results in a 
comprehensive report. The IDOE may specify independent consultants to be used as 
independent facilitators for the alignment studies. The studies will be conducted at a 
convenient, central location in Indiana and will include Indiana educators as the panel 
members. 

2.4.4.4.C.18 Technical Reports 

The Respondent shall prepare a technical report after each year's administration. The 
report should include statistical characteristics of assessment instruments (e.g., validity, 
reliability); design and results of scaling, linking, and standard setting; interpretations of 
assessment results; and other information as requested. The final report will be subject to 
approval by the IDOE. 

2.4.4.4.C.19 Comparability Studies 

At any time should any portion of the summative assessments be administered in more 
than one mode, it will be the responsibility of the Respondent to conduct and report a full 
comparability study. Such a study must determine the impact (if any) of different modes 
of administration and recommend to the IDOE any adjustments to scaling and/or equating 
that may be required. 
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2.4.4.4.C.20 Respondent Burden and Opportunity 



It is the burden of the Respondent to demonstrate whether and how its products and 
services meet NCLB and other legal requirements. The Respondent may utilize proof 
from other states and other sources, or other means, including working with IDOE, to 
meet this burden. 

2.4.4.5 Technical Proposal 

Hie IDOE seeks proposals from Respondents regarding the development (including item 
development and content review), administration (online and paper-and-pencil), standard 
setting, scoring, research and review (comparability studies, rescore study) and collateral 
activities (online scheduling and registration, communication and reporting) of ECAs in 
the following Core 40 courses: 

• Algebra I and H, Geometry, Pre-CalcuIusVTrigonometry, Discrete Mathematics, and 
Probability & Statistics; 

• Biology I, Chemistry I, Physics I, and Integrated Chemistry and Physics; 

• English 9, 10, 11 and 12; 

• U.S. History, U.S. Government, Economics, and World History and Civilization. 

Continued development and use of paper-and-pencil tests for some schools is a practical 
necessity, but this RFP seeks proposals for the online format in all ECAs. 

The IDOE anticipates entering into a multi-year agreement with the Respondent for 
ECAs beginning July 1, 2008, with the option to extend for additional multi-year periods. 

The IDOE requires Respondents to propose a detailed work plan that accomplishes the 
stated objectives. Refining this work plan with the IDOE will be an obligation of the 
successful Respondent, and the revised work plan will be incorporated into the final 
contract documents with the Respondent 

The work plan should permit the Respondent to conduct the necessary preliminary 
activities involved in preparation for the ECAs. The IDOE seeks a workable long-term 
approach to ECAs consistent with the overall assessment system parameters in the Plan. 
The IDOE has provided some specificity about its needs, but Respondents are welcome 
to make suggestions and suggest alternative strategies or approaches (including those 
already in use elsewhere) that could lead to a more effective and efficient assessment 
process. 

The Respondent must identify whether and, if so, how its ECAs correlate to national and 
international standards and other indicators of student performance and whether schools 
and the IDOE will be able to utilize the results of the ECAs to benchmark student results 
in that context. 
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The Respondent should assume both that schools will have independent authority and 
means to assess student writing and that the IDOE will recognize ECAs that assess clarity 
of thought, writing and communication as part of the assessment. 

The Respondent must demonstrate how its ECAs will: 

• Be valid and reliable; 

• Reflect and align to Indiana's Standards; 

• Serve as part of the student's transcript; 

• Enable faculty and schools to monitor individual and collective student results 
longitudinally (for both the students and for the courses) and evaluate and adjust 
instruction to improve student achievement; 

• Predict to success in both preparation for higher education and for career and 
technical pursuits; 

• Give individual teachers feedback that is performance-based and may be used to 
guide, student progress; 

• Enable the results of ECAs to be included in individual student learning portfolios 
and in longitudinal studies of student performance within schools and across the state. 

While not an absolute requirement, the IDOE will consider favorably a program of ECAs 
that also enables teachers to have access to mid-course assessments, curricular 
suggestions and other tools to aid teachers in creating and maintaining rigor, quality, 
consistency and student focus in courses with ECAs; supports research and evaluation 
that can inform professional development and development of instruction; and, 
when combined with course grades and other student information, can inform counseling 
during the student's secondary years and admission, placement and other decisions in 
post-secondary education and employment as part of the student's transcript. 

2.4.4.5.A. Operational Administration of Core 40 End-of-Course Assessments 

End-of-Course Assessment items that have not already been written and pilot-tested and 
operationally deployed must be. The IDOE requires the Respondent to provide all 
additional services necessary to administer these tests in a production or operational 
environment. 

General 

The IDOE expects that tests will have multiple-choice and open-ended item types. For 
purposes of planning and budget preparation, the Respondent should assume that the 
ECAs will involve 70-80% multiple-choice items and 20-30% open-ended items. 

The ECAs administered by the Respondent must be available to all students, and the 
Respondent must propose a plan for appropriate accommodations to all students. 

Although both testing formats will be available to schools during the first multi-year 
agreement, schools will be required to select either the online format or the paper-and- 
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pencil format for each content area. The Respondent will be responsible for the collection 
and organization of this information. 

The IDOE places great importance on test confidentiality, integrity and security. The 
Respondent must propose specific security measures considered appropriate for 
operational administration of the ECAs. 

The state's schools utilize a variety of different academic schedules, including trimesters, 
block scheduling, and a few other variations. Hie Respondent's plan must accommodate 
these alternative schedules, allowing for on-demand testing or a minimum of 3 testing 
windows throughout the school year. 

Hie Respondent must assume access to and use of existing ECA forms, item banks and 
related material where they have been developed in previous pilot and operational tests. 
These materials are the sole property of the State of Indiana, and the Respondent will be 
granted appropriate license and access to these materials. The Respondent must include a 
plan for transition of items, test forms, and student responses. All new materials 
developed as a result of this RFP will be the sole property of the State of Indiana. Where 
the Respondent proposes to utilize forms, item banks or other materials. developed 
elsewhere, the Respondent must grant appropriate and perpetual license and access to 
such materials. 

Online Administration 

The Respondent must be responsible for all operational and support tasks associated with 
administering the online tests of the ECAs, including: 

a. Design and development of the annual test administration cycle and schedule; 

b. Development and publication of web and print instructions, manuals, and other 
communications to schools, including web trainings and online tutorials; 

c. Publication of tests and scoring guides; 

d. Development of an online scheduling system for schools to select test windows and 
delivery formats that will include both testing formats; 

e. Design a mandatory program to assess the technology capabilities of schools and 
respond to various operating systems and browsers; 

f. Development of practice tests (an online set of items that allow students to become 
familiar with online testing and any tools needed by students) and general test 
preparation materials (including an item sampler that provides examples of the item 
types with content that students will see on the test); 

g. Scoring/grading of completed online tests (the fee schedule allows for pricing of 
human scoring or artificial intelligence scoring of open-ended items); 

h. Development of anchor sets and training sets for open-ended items and essays from 
the online and paper-and-pencil pilot pools; 

i. Development of plans for providing accommodations; 

j. Provide appropriate security arrangements for test content and test administration; 
k. Preparation and distribution of online reports for both testing formats to teachers, 
school officials, corporation officials, and the IDOE, with appropriate security 
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arrangements to safeguard access to reports; the Respondent will need to incorporate 

the cut scores into these reports; 
1. Development of promotional materials for use with high schools; 
m. Distribution of electronic communication necessary for registration, testing, 

reporting, and general dissemination of ECA information; 
n. Development of a plan for rescoring essays in cases where schools and the 

Respondent disagree; 

0. Provision of a technical report that includes information regarding administration, 
psychometrics, and summary data for EC As. 

Online field-testing of Algebra I, Algebra II, Biology I, and English 11 items must occur 
to allow for secure item rotation in Spring 2008. The Respondent should assume and 
plan for new and secure item rotation and testing in each successive year's administration 
of the Algebra I, English 1 1 , Algebra II, and Biology I tests (as further described in 
Objective C). The Respondent should provide a separate report and analysis to the IDOE 
on new items introduced in each test administration; this report should include item data 
analysis and any recommendations for improvement of the items. 

The Respondent must deliver to the IDOE aggregate score reports and incorporate the cut 
scores into student, classroom, school, corporation and state-level reports of operational 
test results: The Respondent must deliver a system in which all score reports, both online 
and paper-and-pencil, are delivered in one format to schools, such that scores are 
compiled and shared with schools in the same manner, regardless of the testing format 
selected. Hie IDOE prefers an online system that generates printer-friendly reports at the 
student, classroom, school, corporation and state levels. 

All functions of the online system must be platform, operating system, and browser 
independent. All functions of the online system must work in recent versions of 
Netscape Navigator, Internet Explorer, Firefox, Mozilla, Safari, and Linux, or the 
Respondent must present an alternate plan to address the various platforms, operating 
systems, and browsers used by schools in a secure testing environment 

The Respondent must design and implement a Scoring Reliability (auditing) system for 
the scoring of open-ended and essay items and budget for a reliability system that 
includes a representative sample of student responses. 

Paner-and-Pencil Administration 

1 . The tests' administered by the Respondent must be available to all students, and the 
Respondent must propose a plan for appropriate accommodations to all students. 
Therefore, a large print version of the test will be necessary as well as the possibility 
of a Braille version. 

2. The IDOE requests proposals from the Respondent for scoring of the paper-and- 
pencil tests, including the open-ended and essay items. The Respondent must make 
provisions for the scoring of the paper-and-pencil -based tests by schools or human 
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scoring the open-ended items. The Respondent must score all multiple-choice items 
and consolidate the school-scored open-ended and essay items. In addition to 
providing scoring guides, this will require that schools be able to submit exam results . 
directly to the Respondent for purposes of consolidated score reporting at the student, 
school, corporation and state levels. The Respondent must propose a system for 
scorer training at the school level, including the administration and scoring of tests, 
particularly for those tests that contain open-ended or essay items. 

3. The Respondent must design and implement a Scoring Reliability (auditing) system 
for the scoring of open-ended and essay items, and budget for a reliability system that 
includes a representative sample of student responses. 

4. Hie Respondent will be responsible for all operational and support tasks associated 
with administering the paper-and-pencil tests of the EGAs, including: 

a. Design and development of the annual test administration cycle and schedule; 

b. Development and publication of web and print instructions, manuals, and other 
communications to schools, including web trainings and online tutorials; 

c. Publication of tests and scoring guides; 

d. Development of an online scheduling system for schools to select test windows 
and delivery formats; 

e. Development and execution of a plan for the ordering, delivery, and pick-up of 
paper-and-pencil tests and materials; 

f. Development and publication of practice tests and general test preparation tools; 

g. Scoring/grading of the completed multiple-choice student answer documents, 
which will also include open-ended questions or essay scores provided by 
schools, as well as a scoring reliability check (audit) of teacher-scored tests if not 
human-scored by the Respondent; 

h. Development of plans for providing accommodations; 

i. Provide appropriate security arrangements for test content and test administration; 
j. Preparation and distribution of online reports to teachers, school officials, 

corporation officials and the IDOE, with appropriate security arrangements to 
safeguard access to reports; the Respondent will heed to incorporate the cut scores 
into these reports; 

k. Development of promotional materials (electronic and paper) for use with high 
schools; 

1. Development of training materials for paper-and-pencil scoring of open-ended 
and essay items. 

5. The Respondent must assume and plan for new item rotation and testing in each 
successive year's administration of each ECA. Hie Respondent must provide a 
separate report to the IDOE on new items introduced in each test administration; this 
report must include item data analysis and recommendations, if any, for improvement 
of the items. 

6. The Respondent must deliver to the IDOE aggregate score reports and incorporate the 
cut scores into student, classroom, school, corporation and state-level reports of 
operational test results. The Respondent must deliver a system in which all score 
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reports, both online and paper-and-pencil, are delivered in one format to schools, such 
that scores are compiled and shared with schools in the same manner, regardless of 
the testing format selected. The IDOE prefers an online system that generates printer- 
friendly reports at the student, classroom, school, corporation and state levels. 

Operational Performance Standards for Acceptable Administration 

All Testing 

• Aggregate school or class performance must be reported within 24 hours of the 
completion of the assessment period requested by the school. 

• Aggregate state-level reporting to the IDOE must be reported by June 30 for each 
academic year. 

Computer-Based Testing 

• Students using online testing arrangements must achieve response times meeting 
current industry standards. 

• School officials must receive student test results within 24 hours of test 
completion. 

Paper-and-Pencil Testing 

• Students using paper-and-pencil tests must receive score reports within 7 calendar 
days, starting from receipt of the test by the Respondent. 

2.4.4.5. B. Online Scheduling and Registration, Communication, and Reporting 

System 

The Respondent must provide a system in which: 

1. Corporations can complete online registration for online and paper-and-pencil 
assessments. . 1 

2. The IDOE will have a delivery method for necessary electronic and paper 
communications with corporation and school officials for online and paper-and-pencil 
tests, separately and combined. 

3. An online reporting system for student, school, corporation and state-level data, as 
described in Section A, is available for schools, corporations and the State. 

4. A consolidated help desk will exist to assist corporations with all facets of technology 
issues related to online testing, registration, and reporting, as well as paper-and-pencil 
logistical issues. 
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2.4.4.5.C. Item Development and Content Review 

New items will need to be developed for new operational assessments. Additional items 
will need to be developed periodically. Examples of item types that are desirable in 
layout and format for online and paper-arid-pencil items can be found at: 
http://doe.state.in. us/core40/ . Exact quantities and nature of items appropriate to each 
ECA - multiple-choice, open-ended, essay, numeric response and equation/expression, 
graphing, constructed response or other - will be determined by the IDOE. 

The Respondent will be responsible for conducting passage reviews, content reviews and 
sensitivity/bias reviews, including setting up meetings and facilitating the reviews for 
new EC As. 

Test form construction will be based on item analysis. Hie IDOE anticipates additional 
test forms may be constructed based on field-testing of new items. 

2.4A5.D. Standards (Cut Score) Setting 

The IDOE expects that Standards (cut score) Setting will occur for.each ECA. The 
Respondent must provide the structure, agenda, support, and materials for the Standards 
Setting. The IDOE will manage the logistics and recruit panelists. 

2.4.4.5.E. Comparability Studies 

The IDOE envisions that a comparability study will be completed as each operational 
administration of an ECA concludes. 

2.4.4.5.F. Rescore Study and Report 

A rescoring of open-ended and essay items for online and paper-and-pencil assessments 
must be completed. The Respondent must define rescore samples, perform rescore 
studies, and generate reports of representative samples for all operational assessments. 

2.4.4.6 Cost Proposal 

For purposes of planning and budget preparation, the IDOE expects the number of tests 
administered to vary across schools. For purposes of the Respondent's cost proposal, the 
Respondent should plan for and assume the quantities designated on the formatted cost 
sheet (fee schedule). Attachment E. The IDOE prefers a cost proposal that includes a 
cost structure based on the number of assessments actually administered. 
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2.4.5.1 Background Information 

Indiana currently has no end-of-high-school post-secondary preparation assessment The 
State currently requires students to take the GQE during the 10 grade. 

2.4.5.2 History 

Indiana has not offered or required students to take a post-secondary preparation 
assessment. Indiana has provided funds for schools to offer students the opportunity to 
take tests to gauge their readiness for post-secondary assessments. 

2.4.53 Elements 

Students* K-12 education should prepare them for success in post-secondary education 
and work. The post-secondary preparation assessment is intended to be a guide for 
students, parents, teachers, schools and communities in determining whether each student 
individually, and a school's and community's students collectively, are prepared for that 
success. It is not intended to be a graduation requirement test as such; rather, it is a gauge 
for students as they approach graduation to determine their readiness and preparation for 
success in post-secondary endeavors. 

All students will take the assessment. The assessment will be offered to sophomore 
students as a means to inform them and their parents of their learning status vis-a-vis' 
expectations for success post-high school and to establish a context for academic 
counseling and education and career planning. The assessment will be available to juniors 
and required of all seniors, unless they have achieved a result on the assessment in an 
earlier year that is adequate to define and maximize post-secondary opportunities. 

The assessment will confirm student preparation for success in post-secondary activity, 
whether that is further study or work. For students who have an interest in post-secondary 
education, the assessment will measure preparation for success there. For students who 
have an interest in technical education, the assessment either will be calibrated to also 
gauge readiness for such work, or a different assessment, more compatible with technical 
education, will be offered as a second part of the student's assessment. 

The assessment will be tracked by an individual student's STN, so that the results are 
both portable over geography (as the student moves from school to school or corporation 
to corporation) and available over time (as the student rises through grades K-12 to 
graduation and ultimately into post-secondary education). The assessment will become 
part of each student's learning portfolio, which will evidence the student's progress to 
proficiency in the curricular aims. The assessment results will be reported by school and 
corporation. 
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" The assessment will supplement and not replace the other requirements for a high school 
diploma. The assessment (along with appropriate end-of-course assessments) will replace 
theGQE. 

The assessment will be useful to decisions made by students and their families and for 
counseling and decisions concerning post-secondary directions and aspirations. It will be 
of such a rigor and nature as to be portable across not only post-secondary opportunities 
in Indiana but also those elsewhere in the country. The assessment will be useful to 
decision-making by post-secondary institutions of higher education and employers. It 
will inform their decisions to admit and place students or to hire and place employees. 

2.4.5.4 Technical Requirements 

Respondents must acknowledge their understanding and acceptance of the listed technical 
requirements. Respondents must also provide a narrative for each requirement that 
demonstrates its ability to meet the stated requirement 

Indiana is looking to "offer" an existing instrument or instruments to assist students (and 
parents) in planning for the next phase of their education. Respondents must be specific 
about the intended use and audience for proposed assessments. 

2.4.5.4 .A Demonstrated Acceptance in the "Marketplace" 

Respondents must demonstrate that the assessment is, in fact, accepted and used for the 
purposes by the various institutions that it is targeted to serve. 

2.4.5.4.B Reliability and Validity for Intended Purpose 

Respondents must offer evidence of the reliability, validity and predictive validity of any 
assessment for the represented purpose(s). 

2.4.5.5 Technical Proposal 

The IDOE seeks Respondents of assessments that predict student performance in their 
post-secondary educational endeavors as a means to provide students, parents, teachers, 
counselors and schools with information as to student preparation for success in higher 
education and employment 

Results of such assessments will guide students in course selection, consideration of post- 
secondary options and other personal planning; it will assist teachers, counselors and 
schools in targeting areas for specific student development, specialized work, 
acceleration or remediation. 

Results of the assessments may be used by institutions of post-secondary education in 
making admission and/or placement decisions. In addition, the IDOE seeks proposals for 
assessments that enable national (and, if feasible, international) comparative performance 
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data; that support diagnostic evaluation of student readiness for post-secondary education 
and careers; and that enable students, parents and schools to inform the student's 
individual planning and preparation for graduation and post-graduation pursuits. 

General 

1 . Respondents should assume students in the 1 1* grade and 12 th grade will take the 
assessment. Students in the 10 th grade may take it as well, if the assessment is 
deemed useful to the determination of course selection or other decisions by the 
student at that time. Students in the 12 th grade may elect not to take the assessment if 
they, have previously done so and the results are adequately informative. 

2. Respondents may demonstrate whether the assessment offered may be utilized in 
grades lower than 1 1 th grade and, if so, under what circumstances and to what 
purposes. . 

3. Respondents should specify for cost purposes whether they are prepared to offer the 
IDOE per student, per grade or other pricing. 

4. If the Respondent wishes, it shall clearly mark all information identified concerning 
the assessment that it desires to be maintained confidential as secure - not for 
distribution and also identify a listing of such information in a separate document 
provided to the IDOE. Pricing is not confidential. 

Assessment Dimensions 

1. The Respondent should describe its assessment administration procedures and its 
means and methods for ensuring appropriate administration of the assessment. 

2. Hie Respondent must demonstrate how the assessment utilized will measure student 
. knowledge, skills and abilities consistent with success in post-secondary education 

and employment, including: . 
■ a. How individual performance as assessed correlates to knowledge, skills and 
abilities considered necessary for success in post-secondary education and 
employment; 

b. How the assessment and its results will inform students, parents and schools of 
academic or related needs that the student should address prior to graduation or be 
aware of as the student matriculates to post-secondary education or employment; 

c. How the assessment aligns with Indiana's Academic Standards; 

d. How the assessment aligns with admission, entrance and placement standards in 
post-secondary education; 

e. Whether the assessment (or a separate assessment) includes one or more 
components that measure a student's aptitude and preparation for specific careers; 

f. How the assessment is appropriate for use with students in Indiana; 

g. How the assessment accommodates students who need them; 

h. Whether the Respondent will use a single assessment or multiple assessments in 
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addressing the diversity of post-secondary goals among students; any proposal 
including multiple assessments must specify and explain each individual 
assessment's specific use, targeted student population, validity and reliability 
against that use, score reporting and use, and administration; the Respondent also 
must explain how materials will be distributed, the assessment administered, 
results compiled, and results reported for the individual assessments and, if 
appropriate, collectively. 

3. The Respondent must identify whether and, if so, how the assessment will address the 
student's abilities in writing. 

4. The Respondent should identify and provide any technical manuals or other 
documentation relevant to its proposal, including at least a general description by 
content areas of the technical properties of the underlying scales of the assessment; 
the assessment's scaling procedures; the number and nature of items per content or 
sub-content area; and those areas in which scores are available and the numbers of 
items on which scores are based. 

5. The Respondent must identify in general the design and content of the assessment, its 
usefulness in considerations of instruction and professional development, and whether 
and, if so, how its content areas are linked to and influenced by the requirements for 
success in post-secondary education and employment, specific institutional or general 
predictors of such success, and any current developments in national or international 
content, frameworks, standards or other means to determine student preparation for 
post-secondary success. 

6. The Respondent must describe generally whether and, if so, how it has accomplished 
item development and bias review for the assessment, including race, gender, 
ethnicity, language or other potential impacts on results. 

Administration , - 

1 . The Respondent must propose a plan for the statewide administration of the 
assessment. Hie plan must address the possibility of multiple grade levels (including 
at a minimum 1 1 and 12 th ) taking the assessment. 

2. In administering the assessment, the Respondent must provide sufficient materials or 
other means to administer, score and report the results of the assessment 

3. The Respondent shall demonstrate how it will insure that the assessment as 
administered in Indiana, in each year and over time, will be managed in such a way as 
not to compromise the content of the specific items on the assessment, including 
identifying the security measures used. 

4. The Respondent must demonstrate how it will report results of the assessment, 
including at a minimum by student (including use of the student's STN) and school. 
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Such reporting must enable linkage to the other data systems managed by the IDOE 
and enable inclusion technologically into the student's learning portfolio. 

5. The Respondent must maintain and provide to the IDOE information that confirms 
when each student in a grade has taken the assessment. 

6. The Respondent should indicate the manner and means of administration of the 
assessment, for example, whether it will be administered online, in paper-and-pencil 
format, or otherwise. The Respondent will be responsible for managing the logistics, 
materials and other processes of such administration. Hie Respondent will specify, 
subject to concurrence by the IDOE, the time frames for administration of the 
assessment. The IDOE may assist the Respondent, for example by facilitating 
communication with the schools, but it shall be the Respondent's responsibility to 
arrange for the logistics of the administration of the assessments. Individual schools 
must cooperate with and may not charge the Respondent for use of school space and 
available resources, but the schools may not be required to add additional personnel 
or resources for administration of the assessments. If schools lack sufficient space or 
other logistics, the Respondent may work with schools to arrange off-site 
administration. 

7. Because the results data will publicly released at the school and corporation level, the 
Respondent should address how national reference information including Indiana or 
against which Indiana is evaluated may be impacted by use of this assessment and 
should discuss the stability of comparative data during the time the assessment and its 
results may be used. As part of this, the Respondent should address how the 
proposed instruments or similar instruments are or will be used in other states. The 
IDOE will not consider such use a negative. 

Results and Reporting System 

1. Hie Respondent must identify its turn-around time from the time the student takes the 
assessment(s) until results are available. 

2. The Respondent must identify its methods and timing of reporting and demonstrate 
that such timing and methods match the objectives of the post-secondary preparation 
assessment contemplated by this RFP. At a minimum, the Respondent must 
distribute score reports on the schedule agreed to by the IDOE and must generate 
score reports and analyses for each student and the student's parents and score reports 
(and analyses as suggested by the Respondent in its response) for students, schools, 
corporations, communities or other segments of the overall student population. All 
such reports must tie into the student's STN and enable inclusion of the results and 
analyses in the student's learning portfolio. The Respondent must identify and 
provide samples of the reports it will issue to students, parents, schools and 
corporations and reports that it will create, if asked, concerning other demographics 
(e.g., communities, regions, and the state). Hie Respondent will identify each and all 
items of the reports that will provide normative or comparative reports on a national 
and/or international basis. The Respondent must include in its reports a clear one- 
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page summary report and a companion clear one page guide to understanding the 
results of the report suitable for use by community leaders, corporations, schools, 
teachers, counselors and with students and their parents. 

3. The Respondent should identify its capabilities to report disaggregated and trend data 
at the school, corporation and state levels. 

4. Hie Respondent must identify and describe in detail the nature of reports that it will 
generate concerning student performance on the assessment, including standard and 
optional reports. Hie Respondent also must identify and describe in detail the 
different data sorts that it will provide the IDOE, corporations and schools concerning 
students who take the assessment and their results. 

5. The Respondent must submit to the IDOE a complete data file containing the results 
of the assessment at the same time that it issues the reports. The data file must 
contain all specified information on each student from the assessment(s). This shall 
include, but not be limited to, student identification information that enables 
automated, accurate matching to each student's assessment results and accompanying 
data to the IDOE's student database, utilizing the student's STN. The Respondent 
must transmit the file electronically and securely in a format satisfactory to the IDOE 
and consistent with the IDOE technology environment. 

6. The Respondent should discuss whether it provides any information in addition to 
basic reporting relevant to consideration of classroom instruction, professional 
development, or auricular or other improvement; how such information could be 
presented and used; and how it could help teachers, schools and others effectively and 
efficiently use the basic and any supplemental information for these purposes. 

2.45.6 Cost Proposal 

For purposes of planning and budget preparation, the IDOE expects the number of tests 
administered to vary across schools. For purposes of the Respondent's cost proposal, the 
Respondent should plan for and assume the quantities designated on the formatted cost 
sheet (fee schedule), Attachment E. The IDOE prefers a cost proposal that includes a 
cost structure based on the number of assessments actually administered. 
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2.4.6.1 Background 

As part of their consideration of the Plan, the IDOE and the State Board identified the 
value - to the student, his or her parents, teachers and academic/career counselors - of a 
process to develop individual student learning portfolios throughout the K- 1 2 years. TTie 
portfolios would include, but not be limited to, the diagnostic and summative assessment 
results accumulated by the student through the years. And, depending on the student's 
non-assessment work during K-12 years, the portfolio could evidence a variety of other 
aspects of the student's strengths and interests - writing, creative work, scientific work, 
career and technical work, etc. - all relevant to her or his future decisions. In particular, 
the portfolio would have potential value as the student begins to consider post-secondary 
options during the high school years. The Plan contemplates exploration of such a 
student learning portfolio. 

2.4.6.2 History 

Some Indiana schools have utilized a variety of approaches to student learning portfolios 
over the years, but such efforts have been individual, isolated and sporadic, and the State 
has not provided a common vehicle to support such possibilities across all corporations. 

2.4.63 Elements 

The student learning portfolio system should provide an architecture that electronically 
retains and documents each individual student's assessment results, courses, grades, work 
products, planning and other elements that will assist the student in learning, assist the 
student's teachers and counselors in instruction and guidance over his or her K-12 
educational career, and be available to the student in pursuit of post-secondary learning 
and career opportunities. 

2.4.6.4 Requested Services 

The IDOE solicits Respondent proposals for the electronic student learning portfolio. 

2.4.6.5 Cost Proposal 

For purposes of planning and budget preparation, the IDOE expects the number of tests 
administered to vary across schools. For purposes of the Respondent's cost proposal, the 
Respondent should plan for and assume the public school student quantities (listed 
below). The IDOE prefers Respondents submit a cost proposal that details all costs and 
variables associated with the electronic student learning portfolio by Indiana fiscal year. 
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2.5 COST PROPOSAL 

IDOA requests the pricing associated with this RFP be a firm proposal price that must 
remain open and in effect for a period of not less than 180 days from the proposal due 
date. Respondents must provide separate cost proposals for each component contained in 
the Respondent's proposal. The following elements are common to all components: 

Discounts for multi-component award: The State will evaluate favorably those 
Respondents who propose a discounted cost proposal for being awarded more than one 
component. Respondents should clearly indicate that they are not willing to offer a 
discount, or provide details into the discount structure and application of such discounts. 
Respondents should provide an example that illustrates the application of the offered 
discount structure. 

Performance Holdbacks: The State will evaluate favorably those Respondents who 
propose performance holdbacks. Essentially, the State would only pay the Respondent 
for a percentage of the contractual fee. Hie remaining percentage would be paid once 
both parties agree that the Respondent succeeded in meeting pre-defined, objective, 
measurable goals. Respondents should clearly indicate that they are not willing to offer a 
performance holdback, or provide details into the objective and measurable goal(s) to be 
obtained and any other suggested details. Respondents must clearly indicate to which 
component the performance holdback applies. 

Respondents must provide prices for each Indiana Fiscal year (Julyl to June 30). In 
addition, prices must be provided separately for each component. Respondents must use 
Attachment E for the cost proposal and maintain the structure and format of that 
document. The only exception to this requirement is Component 6, Respondent Cost 
Proposal. 

2.6 INDIANA ECONOMIC IMPACT 

All companies desiring to do business with state agencies must complete an "Indiana 
Economic Impact" form (Attachment D). The collection and recognition of the 
information collected with the Indiana Economic Impact form solicits information 
concerning the economic impact a project will have on Indiana and its residents 
regardless of where a business is located. The collection of this information does not 
restrict any company or firm from doing business with the state. 

Respondents must submit separate Indiana Economic Impact forms for each 
component included in Respondents proposal. The information reported by the 
Respondent within the Indiana Economic Impact should be exclusive to that 
component's Indiana Economic Impact 
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2.7 BUY INDIANA INITIATIVE/INDIANA COMPANY 



It is the Respondent 1 s responsibility to confirm its Buy Indiana status for this portion of - 
the process. If a Respondent has previously registered its business with IDOA, go to 
www.BuvIndiana.In.gov and click on the link to update this registration. Click the tab 
titled Buy Indiana. Select the appropriate category for your business. Respondents may 
only select one category. Certify this selection by clicking the check box next to the 
certification paragraph. Once this is complete, save your selection and exit your account 

Respondents that have not previously registered with IDOA must go to 
www.BuvIndiana.IN.gov and click on the link to register. During the registration 
process, follow the steps outlined in the paragraph above to certify your business* status. 
The registration process should be complete at the time of proposal submission. 

Defininp; an Indiana Business: 

"Indiana business" refers to any of the following: 

(1) A business whose principal place of business is located in Indiana. 

(2) A business that pays a majority of its payroll (in dollar volume) to residents of 
Indiana. 

(3) A business that employs Indiana residents as a majority of its employees. 

Respondents claiming this status must indicate which of these provisions cited 
immediately above qualifies them as an Indiana business. They must also fully complete 
the Indiana Economic Impact Form (Attachment C) and include it with their response. 

The following is the policy concerning items 4 & 5 described below. Appropriate 
documentation must be provided with your proposal response supporting either claim 
made below: 

(1) A business that makes significant capital investments in Indiana. 

(2) A business that has a substantial positive economic impact on Indiana. 

Substantial Capital Investment : Any company that can demonstrate a minimum 
capital investment of $5 million or more in plant and/or equipment or annual lease 
payments of $2.5 million or more shall qualify as an Indiana business under category #4. 
If an out of state company does not meet one of these criteria, it can submit 
documentation/justification to the State for review for inclusion under this category. 

Substantial Indiana Economic Impact : Any company that is in the top 500 companies 
(adjusted) for one of the following categories: number of employees (DWD), 
unemployment taxes (DWD), payroll withholding taxes (DOR), or Corporate Income 
Taxes (DOR); it shall qualify as an Indiana business under category #5. If a Respondent 
needs assistance in determining if its business, qualifies under this criterion, please send 
an email inquiry to buyindianainvest@idoa.in.gov and you will receive a response within 
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forty-eight (48) hours. If an out of state company does not meet one of these criteria, it 
can submit documentation/justification to the State for review for inclusion under this 
category. 

Pursuant to IC 5-22-15-7, Respondents may claim only one preference. For the purposes 
of this RFP, this limitation to claiming one preference applies to Respondents* ability to 
claim the recycled content preference, Indiana small business preference, or to claim 
eligibility for Buy Indiana points. 
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SECTION THREE 
PROPOSAL EVALUATION 



3.1 PROPOSAL EVALUATION PROCEDURE 

The State has selected individuals to act as a proposal evaluation team. Subgroups of this 
team, consisting of one or more team members, will be responsible for evaluating 
proposals with regard to compliance with RFP requirements. All evaluation personnel 
will use the evaluation criteria stated in Section 3.2. ITie Commissioner of IDOA or her 
designee will, in the exercise of her sole discretion, determine which proposals offer the 
best means of servicing the interests of the State. The exercise of this discretion will be 
final. 

The procedure for evaluating the proposals against the evaluation criteria will be as 
follows: 

Each proposal will be evaluated for adherence to requirements on a 
pass/fail basis. Proposals that are incomplete or otherwise do not conform 
to proposal submission requirements may be eliminated from 
consideration. 

Each proposal will be evaluated on the basis of the categories included in 
Section 3.2. A point score has been established for each category. 

If technical proposals are close to equal, greater weight may be given to 
price. 

Based on the results of this evaluation, the qualifying proposal determined 
to be the most advantageous to the State, taking into account all of the 
evaluation factors, may be selected by IDOA, the State Board and the 
IDOE for further action, such as contract negotiations. If, however, IDOA, 
the State Board and the IDOE decide that no proposal is sufficiently 
advantageous to the State, the State may take whatever further action is 
deemed necessary to fulfill its needs. If, for any reason, a proposal is 
selected and it is not possible to consummate a contract with the 
Respondent, IDOA may begin contract preparation with the next qualified 
Respondent or determine that no such alternate proposal exists. 

3.2 EVALUATION CRITERIA 

Proposals will be evaluated based upon the proven ability of the Respondent to satisfy the 
requirements of the RFP in a cost-effective manner. Each of the evaluation criteria 
categories is described below with a brief explanation of the basis for evaluation in that 
category. The points associated with each category are indicated following the category 
name (total maximum points = 100). If any one or more of the listed criteria on which 




3.1.1 

3.1.2 
3.1.3 
3.1.4 
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the responses to this RFP will be evaluated are found to be inconsistent or incompatible 
with applicable federal laws, regulations or policies, the specific criterion or criteria will 
be disregarded and the responses will be evaluated and scored without taking into 
account such criterion or criteria. 



Summary of Evaluation Criteria for each component: 





Itf Points H Jjpl 


1. Adherence to Mandatory Requirements 


Pass/Fail 


2. Management Assessment/Quality (Business and Technical Proposal ) 


30 


3. Cost (Cost Proposal) 


25 


4. Indiana Economic Impact 


15 


5. Buy Indiana 


10 


6. Minority (10) and Women Business (10) Subcontractor Commitment 


20 







All proposals will be evaluated using the following approach. 
Step 1 

In this step proposals will be evaluated only against Criteria 1 to ensure that they adhere 
to Mandatory Requirements. Any proposals not meeting the Mandatory Requirements 
will be disqualified. 

Step 2 

The proposals that meet the Mandatory Requirements will then be scored based on 
Criteria 2 and 3 ONLY. This scoring will have a maximum possible score of 55 points. 
Ail proposals will be ranked on the basis of their combined scores for Criteria 2 and 3 
ONLY. This ranking will be used to create a "short list" Any proposal not making the 
"short list" will not be considered for any further evaluation. 

Step 2 may include one or more rounds of proposal discussions focused on cost and other 
proposal elements. 

Step 3 

The short-listed proposals will then be evaluated based on all of the evaluation criteria 
outlined in the table above. 
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If the State conducts additional rounds of discussions and a Best & Final Offer (BAFO) 
round which lead to changes in either the technical or cost proposal for the short-listed 
Respondents, their scores will be recomputed. 

The section below describes the different evaluation criteria. 



3.2.1 Adherence to Requirements - Pass/Fail 

3.2.2 Management Assessment/Quality - 30 points 

3.2.3 Price - 25 points 

3.2.4 Indiana Economic Impact ( 1 5 points) 

See Section 2.6 for additional information. 

The total number of full time equivalent (FTE - please see Section 1 .2 for 
a definition of FTEs) Indiana resident employees for the Respondent's 
proposal (prime contractor and subcontractors) will be used to evaluate the 
Respondent's Indiana Economic Impact. Points will be awarded based on 
a graduated scale. The Respondent with the most Indiana FTEs will be 
awarded IS points. Points will then be awarded to the remaining 
Respondents proportionately. 

3.2.5 Buy Indiana Initiative - 10 points 



Respondents qualifying as an Indiana Company as defined in Section 2.7 
will receive 10 points in this category. 

3.2.6 Minority (10 points) & Women's Business (10 points) Subcontractor 
Commitment - 20 points. 

The following formula will be used to determine points to be awarded: 

The commitment factor for each proposal will be calculated by 
multiplying the commitment percentage by one hundred. The RFP score 
ration will be determined by dividing the maximum allowable points by 
the highest commitment factor. The proposal with the highest 
commitment factor will be given the maximum allowable points. The 
points awarded to the other proposals will be calculated by multiplying the 
score ratio by the proposed commitment factor. 

Commitment percentage * 100 = commitment factor 

Maximum allowable points/highest commitment factor = score ratio 

Commitment factor * score ratio = points awarded 
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The Commissioner of IDOA or her designee will, in the exercise of her sole discretion, 
determine which proposal(s) offer the best means of servicing the interests of the State. 
The exercise of this discretion will be final. 

Hie IDOE requests that all Respondents interested in offering services submit a response 
in the manner described here. Based upon me responses and, if the IDOE determines that 
such responses indicate that the goals of the Assessment System can best be served by 
contracting with one or more Respondents, the IDOE shall select one or more Respondents 
who can best satisfy the needs of the System. 

This is an invitation to bid and a Request for Proposals under the Indiana Procurement 
Code (IC 5-22-9). The receipt of any proposals submitted pursuant hereto creates no 
. obligation, expressed or implied, on the part of the IS BE, the IDOE or the State of Indiana. 
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